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I.  INTRODUCTION 

A.     MOTIVATION  FOR  THIS  THESIS 

As  computing  systems  perform  more  and  more  sophisticated  functions, 
the  software  components  of  such  systems  necessarily  become  larger  and 
more  complex.  In  addition,  a  plethora  of  critical  systems,  such  as  nuclear 
power  plants  and  aircraft  control  systems  are  controlled  by  software.  The 
growth  in  size  and  complexity  of  software  results  in  an  even  faster  growth 
in  the  complexity  of  software  testing.  Testing  is  currently  a  very  labor 
intensive  and  expensive  process  that  accounts  for  approximately  50%  of 
software  system  development  (Myers  1979,  Korel  1990). 

The  need  for  reliable  software  operation  is  increasing  rapidly.  This 
implies  that  extensive  software  testing  is  frequently  necessary  despite 
expenses.  This  thesis  deals  with  software  testing,  and  is  intended  as  a  sequel 
to  previous  research  on  failure  region  identification  (Shimeall  et  al  1991)  and 
failure  region  clustering  (Ginn,  1991). 


B.     OUTLINE  OF  THE  PROBLEM 

Failure  regions  appear  to  have  a  tendency  to  cluster  so  the  neighbo- 
rhood of  a  specific  failure  region  may  reveal  more  software  faults  than  the 
one  that  caused  the  region.  An  empirical  study  (Ginn  1991)  on  data  obtained 
by  Shimeall  and  Leveson  (1991)  has  demonstrated  this  clustering  tendency 
of  failure  regions  using  appropriate  clustering  criteria  for  multidimensional 
nominal  types  of  data  (Jain  and  Dubes,  1988).  A  number  of  issues  have  been 
raised  from  this  research: 


•  Several  failure  regions  demonstrated  a  strong  clustering  tendency  in  one 
dimension  but  weak  clustering  in  other  dimensions.  However  it  is  not 
known  if  this  is  a  coincidence  or  normal  behavior  of  failure  regions. 

•  Software  faults  were  numbered  in  order  as  they  were  discovered,  by  the 
various  testing  techniques  applied  by  Shimeall  and  Leveson  (1991),  so 
that  many  of  the  sequentially  numbered  faults  were  discovered  by  the 
same  detection  technique.  There  was  also  a  strong  tendency  of 
clustering  for  sequential  faults,  but  it  is  as  yet  unknown  if  there  is  a 
correlation  between  certain  detection  techniques  and  certain  types  of 
fault  clusters. 

•  It  is  not  clear  which  types  of  conditions  and  variables  are  more  likely 
to  result  in  clusters. 


There  is  empirical  evidence  that  known  failure  regions  may  be  used  to 
understand  the  relationship  of  one  fault  to  another.  Failure  regions  offer  a 
mechanism  for  identifying  common  features  among  faults,  because  the 
relationship  between  two  failure  regions  corresponds  to  the  relationship  of 
the  code  locations  of  the  associated  faults.  The  goal  of  this  research  will  be 
an  improved  testing  technique  that  incorporates  failure  region  behavior.  To 
do  this,  we  need  to  better  understand  how  parts  of  the  program  interact, 
since  faults  with  similarities  in  their  failure  regions  are  expected  to  occur 
under  similar  conditions. 

C.     OVERVIEW  OF  THE  THESIS 

Chapter  II  gives  an  extensive  review  of  preceding  research  on  software 
testing  in  general  and  failure  region  analysis  in  particular.  Chapter  III 
introduces  a  testing  technique  on  clustering  and  presents  empirical  results 
from  the  application  of  the  said  technique  on  the  results  of  a  software 
experiment.  Finally,  Chapter  IV  summarizes  the  conclusions  that  can  be 
drawn  from  the  results  and  offers  suggestions  and  recommendations  for 
ftxrther  research. 
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n.  BACKGROUND  AND  RELATED  WORK 

This  chapter  reviews  software  testing  definitions  and  methods  for 
dealing  with  software  fault  and  failure  association.  Symbols  and  terminology 
are  similar  to  those  used  in  the  classical  papers  (such  as  Weyuker  and 
Ostrand,  1980,  Goodenough  and  Gerhart,  1975).  A  brief  overview  of  theories 
of  software  testing  related  to  our  research  is  presented.  Previous  work,  both 
empirical  and  theoretical,  in  the  area  of  failure  regions  analysis  is  included. 
Finally  the  cluster  analysis  technique  that  has  been  used  by  Ginn  (1991)  is 
briefly   presented. 

A.     SOFTWARE  TESTING 

1.      Basic  Definitions 

Testing  is  a  method  of  program  verification  that  deduces  from 
execution  that  a  program  possesses  required  properties  (Morell  1990). 

The  input  data  for  the  majority  of  programs  come  from  a  multi- 
dimensional space.  An  example  given  by  Amman  and  Knight  (1988)  refers 
to  a  program  processing  an  input  of  20  floating  point  numbers  thus  having 
20  dimensions. 
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We  shall  use  D  for  the  input  domain  of  a  program  F,  and  R  for  the 
output  range  of  F.  On  input  d  (dED)  F,  if  it  terminates,  produces  output 
F(d)  (gR).  Hence  the  program,  may  be  viewed  as  a  mapping  from  the  input 
domain  to  the  output  range. 

The  output  specification  for  F  is  given  by  OUT(x,y),  where  xgD 
and  yeR  F  is  correct  on  input  d  (abbreviated  OK(d))  if  F(d)  exists  and 
OUT(d,F(d)).  A  test  T  (c  D)  is  formally  defined  as  a  subset  of  the  input 
domain  (after  Weyuker  and  Ostrand,  1980,  Goodenough  and  Gerhart,  1975). 
More  often  than  not,  software  failures  appear  at  what  seems  to  be  an  obscure 
set  of  data  or  a  special  case  (Ammann  and  Knight  1988).  This  introduces  an 
additional  complication  to  the  testing  process. 

2.      Ideal,  Valid,  and  Reliable  Tests 

An  ideal  test  must  be  valid,  which  implies  that  for  every  fault  in 
program  F  exist  entries  in  test  T  that  cause  the  fault  to  execute  and  produce 
incorrect  output  (Goodenough  and  Gerhart  1975).   Formally. 

Ideal{T)-(V Fault eF:  3  deTA  ^OK(d))  (2.1) 

A  reliable  test  is  one  that  either  produces  entirely  correct  output  or 
entirely  incorrect.  Formally: 


Reliable{T)-[(ydeT:  OKid))\/(yd€T:  ^OK(d))]  (2.2) 

The  importance  of  a  failure  region  clustering  theory  becomes  apparent 
here  from  the  point  of  view  of  test  selection.  It  was  shown  by  Weyuker  and 
Ostrand,  (1980)  that  an  ideal  and  reliable  test  is  exhaustive  therefore  it  is 
desirable,  for  the  sake  of  feasibility,  to  select  a  much  smaller  test  to  the  end 
of  revealing  certain  types  of  faults.  A  failure  region  clustering  theory 
provides  same  guidelines  for  this  type  of  selection. 

B.     PROGRAM  PATHS 

1.      The  Concept  of  a  Program  Path 

A  path  in  a  program  is  defined  as  a  sequence  of  statements  (Rich- 
ardson and  Clarke  1985).  It  is  quite  obvious  that  a  path  may  be  decomposed 
into  a  number  of  subsidiary  paths.  A  block  statement  is  decomposed  into  its 
constituent  enclosed  statements  whose  execution  depends  upon  evaluation  of 
a  condition  which  is  also  part  of  the  block  statement. 

The  control  flow  statements  in  a  computer  program  partition  the 
input  space  into  a  set  of  mutually  exclusive  domains  each  of  which  causes 
a  corresponding  path  to  be  executed  (White  and  Perera  1986).  This  concept 
of  path  offers  a  natural  way  to  partition  the  input  domain. 


The  subdomain  Dj  of  path  j,  is  defined  by  a  boolean  expression 
(say  P(j))  that  is  the  conjunction  of  the  path's  branch  predicate  constraints. 
In  general  these  predicates  are  expressed  in  terms  of  both  local  (program) 
and  input  variables.  However  it  is  possible  to  replace  each  program  variable 
appearing  in  the  predicates  by  its  symbolic  value  defined  in  terms  of  input 
variables  along  that  path  and  get  an  equivalent  constraint  that  is  the  partition 
boundary,  P(j),  as  a  function  of  input  variables  only  (also  predicate 
interpretation  after  White  and  Wiszienwski  1988). 

2.      Programs  as  Sets  of  Partial  Functions 

Using  the  concept  of  path,  as  defined  in  the  previous  subsection, 
we  can  model  a  program  as  a  set  of  partial  functions  from  the  input 
partitions  (Dp  to  the  output  space  (O),  each  of  the  partial  functions 
corresponding  to  the  execution  of  a  sequence  of  statements  along  the 
corresponding  path  (j)  (Richardson  and  Clarke  1985).  Formally: 

Sj.,S,^..  VD^  =  O'eO  (2.3) 

where  Sj  ,Sj  2..Sj  „  stands  for  the  sequence  of  statements'  along  path  j.  In  the 
case  of  loops  executing  along  the  path  (let  Sj,Sj,+,..Sj,+n,  be  the  sub  sequence 


*        The  condition  evaluations  are  included  into  the  sequence  of  statements  for 
consistency  although,  since  the  path  is  pre-determined  they  do  not  affect 
the  result  of  the  operation. 


of  statements  in  the  loop  body)  the  path  sequence  of  statements  may  be 
written  Sj jSjjl  Sj|Sj,+j..Sjj+„)''..Sj„.  This  model  assigns  sequences  that  differ 
only  by  their  number  of  loop  iterations  to  the  same  program  path. 

As  a  path  may  be  decomposed  into  other  paths,  an  input  domain 
partition  may  be  refined  by  the  same  token  in  a  hierarchical  way,  as  new 
branch  predicate  conditions  are  "AND  ed"  to  the  existing.  This  suggests  a 
more  generic  approach  to  the  issue  of  input  domain  partition  than  the  one 
suggested  by  Richardson  and  Clarke  (1985)  where  they  distinguish  this 
partition  into  implementation  and  specification  partitions  and  point  out 
certain  discrepancies  due  to  the  inherent  differences  of  the  specification 
versus  the  implementation  languages. 

3.      Regular  Expressions  for  Paths 

The  set  of  paths,  on  a  flow-chart,  can  be  expressed  in  algebraic 
form  (Beizer  1990).  Path  expressions  are  converted  to  regular  expressions 
that  can  be  used  to  examine  structural  properties  of  program  paths. 

For  any  single  path,  j,  of  the  program,  the  sequence  of  statements 
Sj,Sj2-Sjn  introduced  in  the  previous  section  is  \ht  path  product  of  j.  Path 
products  are  also  defined  on  path  segments.  The  path  that  consists  of 
successive  path  segments,  has  a  path  product  equal  to  the  concatenation  of 

8 


their  path  products.  A  set  of  parallel  paths  between  two  nodes  has  a  path 
product  equal  to  the  sum  of  the  path  products  of  the  parallel  paths. 

Condition  evaluations  are  retained  in  the  path  products  for 
consistency  (cf  previous  subsection).  In  fact,  the  path  products  of  the  two 
segments  starting  from  a  decision  point  (such  as  an  if  C  then  else  construct 
or  a  loop  exit  condition  etc)  are  preceded  with  the  condition  C  and  condition 
^C  respectively,  depending  upon  which  part  of  the  decision  (if  or  else) 
results  in  their  execution.  The  regular  expressions  for  paths  provide  a  concise 
and  compact  notation  for  both  the  path  conditions  (which,  for  a  given  set  of 
paths,  are  derived  by  conjunction  of  all  conditions  on  successive  path 
segments  and  disjunction  of  conditions  on  parallel  path  segments)  and  path 
actions  (which,  in  the  previous  section,  are  modelled  as  partial  functions 
from  the  input  domain  to  the  output  range). 

C.     ON  ERRORS  AND  FAULTS 

A  fault  is  an  erroneous  piece  of  program  source  code,  while  an  error 
is  a  discrepancy  between  a  computed  value  and  the  true,  specified,  or 
theoretically  correct  value. 


A  failure  specifies  the  inability  of  a  module  to  perform  its  specified 
ftinction  and  includes  both  erroneous  output  and  failure  to  produce  output 
(see  ANSI-IEEE  STD  610.12-1990). 

A  fault  (E)   is  formally  modeled  as  a  3-tuple  (Shimeall  et  al  1991): 

E  =  <L^V^C  >  (2.4) 

where  L^  is  the  location  of  the  fault^  (which  is  some  program  statement),  V, 
the  list  of  variables  that  form  the  error  caused  by  the  fault  and  Q  is  a 
Boolean  condition  under  which  the  fault  is  activated.  An  interesting  point  is 
that  a  fault  activation  does  not  always  imply  a  failure  (coincidental 
correctness.  Morel  1988).  There  are  three  conditions  that  must  hold  true  for 
a  fault  to  produce  a  failure  (Shimeall  et  al  1991)  which,  apart  from  Cp  are 
the  reachability  condition  and  the  error  propagation  condition. 

A  basic  assumption  about  the  faults  in  code  is  the  competent  program- 
mer hypothesis  (DeMillo  et  al  1978).  This  states  that  a  competent  program- 
mer will  write  a  program  that  is  syntactically  close  to  the  correct  program, 

A  taxonomy  of  the  types  of  faults  possibly  found  in  a  program  is  given 
by  Reiser  (1990): 


Sometimes  a  fault  is  "distributed"  to  more  than  one  locations  while 
certain  types  of  faults,  such  as  missing  functions  etc. ,  do  not  have  a  well 
defined  location. 
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1.  Requirements  and  Specifications 

These  include  incomplete  or  self-contradictory  specifications, 
missing,  wrong  or  superfluous  features,  and  not-well-specified  feature 
interactions. 

2.  Structural  Errors 

Such  as  control  and  sequence  errors,  logic  errors,  incorrect 
formulae  applications,  use  of  uninitialized  variables.  These  can  be  further 
divided  (Richardson  and  Clarke  1985)  into: 

a.  Computation  Errors 

A  computation  error  occurs  when  the  correct  path  through  the 
program  is  taken,  but  the  output  is  incorrect  because  of  faults  in  the 
computation  along  the  path. 

b.  Domain  Errors 

A  domain  error  (White  and  Perera  1986)  occurs  when  a  specific 
input  follows  the  wrong  path  due  to  an  error  in  the  control  flow  of  the 
program.  They  are  of  two  kinds: 
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(1)  Missing  Path  Errors.  They  occur  when  a  special  case 
requires  a  unique  sequence  of  actions,  but  the  program  does  not  contain  a 
corresponding  path. 

(2)  Path  Selection  Errors.  They  occur  when  the  program 
recognizes  the  need  for  a  path,  but  incorrectly  determines  the  conditions 
under  which  the  path  is  executed. 

3.  Data  Errors 

Occur  when  a  specific  input  follows  the  correct  path,  but  an  error 
such  as  wrong  data  declarations,  wrong  data  initialization  (especially  in 
shared  dynamic  objects),  etc.  results  in  erroneous  output. 

4.  Coding  Errors 

The  most  common  coding  errors  are  documentation  inconsistencies, 
typographical  errors,  and  erroneous  use  of  a  program  statement  when  its  side 
effects  are  not  well  understood. 

5.  Interface  Eitdis 

Such  as  interface  communication  problems,  incorrect  input-output 
format,  wrong  subroutine  control  sequence,  wrong  call  parameters,  incon- 
sistent entry  or  exit  parameter  values,  etc. 
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D.     ON  FAILURE  REGIONS 

A  software  failure  region  (GcD)  is  the  set  of  all  input  values  that  are 
mapped  by  an  individual  program  fault  onto  any  failure  (or  onto  a  failure  set, 
as  in  figure  1.1).  It  is  noted  that  the  concept  of  failure  region  includes  both 
the  input  points  which  cause  erroneous  output  and  the  geometry  of  it.  These 
sets  are  always  finite,  since  the  number  of  representations  in  a  fmite  machine 
is  limited,  but  more  often  than  not  intractably  large.  The  failure  region  boun- 
daries are  defined  by  Boolean  conditions  on  the  input  domain. 

Shimeall  et  al  (1991)  demonstrate  a  technique  that  analytically  de- 
termines the  three  conditions  for  a  known  fault  (which  include  all  conditions 
for  reaching,  activating  and  propagating  the  fault),  by  symbolically  executing 
(King  1976)  the  source  code  on  every  "loop  [0,1]"  path  (Loops  are  handled 
as  in  previous  section,  by  applying  either  the  exit  condition  or  the  loop 
effects)  and  conjuncting  the  obtained  Boolean  expressions.  The  conjunction 
of  those  conditions  is  the  mathematical  specification  of  the  failure  region 
boundary  (which  may  be  called  Boiuid(G))  subject  to  the  limitations  of  finite 
representation  in  computing  machines. 
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Figure  1.1  Associations  between  Failure  regions  Faults 

and  Failures  (adapted  from  Shimeall  et  al 
1991) 


E.      BOUNDS    CORRELATION    BETWEEN    DIFFERENT    FAILURE 


REGIONS 


The  dimensions  of  the  input  domain  of  a  program  can  provide  a  set  of 
criteria  for  the  correlation  between  different  failure  regions.  To  this  end  a 
classification  of  these  dimensions  is  used  (Ginn  1991),  with  respect  to  any 
pair  of  failure  regions: 
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The  dimensions  that  appear  in  both  boundaries  in  exactly  the  same  way 
are  termed  identically  participating  dimensions. 

Those  who  appear  in  both  boundaries  but  not  in  an  identical  way  are 
termed  coincidentally  participating  dimensions. 

The  dimensions  that  do  not  appear  in  the  boundaries  of  both  regions  are 
called  nonbounding  dimensions,  because  the  boundaries  these  dim- 
ensions place  on  failure  regions  are  no  more  restrictive  than  their  entire 
range  of  values. 


In  Ginn  (1991)  the  Identical  and  Coincidental  dimensions  are  collective- 
ly referred  to  as  Composite  dimensions. 
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ID.  CLUSTER  ANALYSIS  AS  A  PREDICTOR  FOR   FAULTS 

A.     CLUSTER  ANALYSIS 

The  identification  of  groups  that  have  similar  characteristics  is  the  goal 
of  clustering  analysis.  In  the  context  of  this  thesis,  the  objective  of  failure 
region  analysis  is  to  investigate  clustering  of  failure  regions  and  identify  the 
logical  relation  of  the  program  locations  of  the  faults  responsible  for  these 
failure  regions. 

By  definition  of  failure  region  (Shimeall  et  al  1991  for  example)  the 
execution  of  a  program,  with  data  from  some  failure  region  as  input,  will 
reach,  activate,  and  propagate  to  the  output  the  corresponding  program  fault. 
The  conjunction  of  predicates  that  defines  the  bound  of  the  region  (Chapter 
II-D)  contains  a  statement  of  the  reachability,  activation  and  propagation 
conditions  for  the  corresponding  fault. 

The  criterion  used  by  Ginn  (1991)  assumed  that  clustering  between 
failure  regions  occurs  when  identical  or  similar  predicates  appear  in  the 
bounds  of  these  failure  regions.  Therefore,  one  could  use  identical  dim- 
ensions (corresponding  to  identical  predicates)  or  composite  dimensions 
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(corresponding  to  both  identical  and  similar  predicates)  to  define  a  measure 
of  clustering.  The  assumption  made  by  Ginn  (1991)  is  reasonable,  because 
it  is  expected  that  when  failure  regions  correspond  to  faults  that  share  some 
program  paths  (therefore  some  of  the  predicates  in  common)  it  is  possible, 
but  not  necessary,  that  the  same  input  may  reveal  all  of  them.  On  the 
contrary,  when  the  faults  do  not  share  any  path,  it  is  impossible  for  both  of 
them  to  be  revealed  by  the  same  input. 

The  criteria  used  for  the  failure  region  bounds  variables  are  essentially 
ordinal:  identical,  coincidental,  nonbounding,  in  descending  order.  Therefore 
relative  values  cannot  be  assigned  to  them.  There  are,  however,  two  different 
coefficients  that  may  serve  as  a  measure  of  clustering  for  ordinal  data  (see 
Jain  and  Dubes  1988): 


S(G,Gp  =  ^^^^  (3.1) 


The  simple  matching  coefficients  for  two  failure  regions  Gj,  Gj  are 
defined  by  3.1,  where  a^o  stands  for  the  non-bounding  dimensions  of  both 
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failure  regions  a,,  for  the  shared^  dimensions  of  Gi  and  Gj,  (cf  Chapter  II-E) 
and  a  for  the  total  number  of  input  dimensions.  The  similarity  of  failure 
regions  Gj  and  Gj  is  greater  the  closer  the  simple  matching  coefficient  is  to 
unity,  since  S(Gi,Gi)=l,  because  ao,=a,Q=0.  The  inclusion  of  ao^  in  the 
numerator  of  the  coefficient  emphasizes  equally  bounding  and  non-bounding 
dimensions  and  can  result  in  very  high  (close  to  unity)  values  of  the 
coefficient  when  the  total  number  of  input  dimensions  is  much  greater  than 
the  bounding  dimensions  of  two  failure  regions.  Suppose  for  example  the 
pair  of  failure  regions: 

Bound(G^)  =ix<5)  My  <i)Aiz<  10)  (3.2) 


BoundiG^  =  (>v<0)A(w<0)  (^S^ 

If  the  total  number  of  dimensions  is  250  then  3^0=245,  a,,=0,  and 
S(G„G2)=245/250=0.98,  which  implies  a  high  degree  of  clustering  while  it 
is  obvious  that  the  failure  regions  are  not  related. 


The  term  shared  may  refer  to  dimensions  appearing  in  identical  predicates  in  two 
failure  regions  definitions,  or  may  include  both  similar  and  identical  predicates.  Ginn 
(1991)  investigates  both  cases  separately. 
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The  Jaccard  coefficients,  defined  by  3.4,  seem  more  sensitive  to 
clustering  since  they  emphasize  more  the  composite  dimensions,  as  opposed 
to  the  simple  matching  coefficients  that  are  symmetric  in  composite  an  non- 
coincidental  dimensions.  As  with  the  simple  matching  coefficients,  the 
similarity  of  failure  regions  Gj  and  Gj  is  greater  the  closer  the  Jacard 
coefficient  is  to  unity,  since  J(Gi,Gi)=l. 


a 
a -a 


J{G,Gj)  =  —^  (3.4) 


00 


However,  under  certain  circumstances,  the  reliability  of  Jaccard 
coefficients  is  debatable.  Suppose  for  example  the  pair  of  failure  regions 
defined  by  (3.5),  (3.6): 

Bound{G^)={x<5)My<i)A{z<lO)  (3.5) 


Bound(G,)  =  (x<5)^(y<S)^{z<lO)^{w<0)^(u<0)        (3.6) 
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In  this  case  the  Jacard  coefficient  J(G„G2)=3/(3+2-H))=0.6,  while  it  is 
intuitively  obvious  that  the  relation  is  probably  stronger  than  is  suggested  by 
the  coefficient.  On  the  other  hand,  the  simple  matching  coefficient, 
depending  upon  the  number  of  dimensions  that  are  non-bounding  for  both  G, 
and  Gj  may  vary  between  0.6  and  0.9999. 

In  the  following  sections,  we  shall  examine  how  pairs  of  faults  may 
result  in  clustered  failure  regions,  as  well  as  how  the  Jaccard  coefficients  are 
affected. 

B.     RELATED  FAULTS 

The  goal  in  analyzing  fault  relations  is  to  understand  the  clustered 
failure  regions,  which  appear  in  software  experiments  (Ginn  1991).  To  this 
end,  it  is  required  to  determine  what  kind  of  fault  relationship  results  in 
clustered  failure  regions. 

L      Logical  Relation  of  Faults 

Faults  can  be  logically  related  if  they  are  either  the  same  logical 
flaw  {Taxonondcally  related)  or  they  are  located  in  regions  of  programs  that 
compute  the  same  part  of  the  application  {Functionally  Related). 
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2.      Taxonomically  Related  Faults 

A  logical  flaw  is  an  error,  or  misunderstanding  by  the  programmer, 
in  the  design  logic  of  a  program  that  results  in  a  number  of  faults.  Taxo- 
nomically related  faults  are  expected  to  be  both  application  dependent  and 
programmer  dependent.  They  are  application  dependent  because  it  is 
expected  that  applications  requiring  a  great  number  of  a  certain  type  of 
constructs  (say  loops  or  if  statements  for  example)  are  more  prone  to  the 
type  of  faults  peculiar  to  these  constructs  (loop  iteration  and  control  flow 
faults  respectively  for  this  example).  It  is  not  always  clear  from  the 
specification  of  a  program  exactly  how  many  such  constructs  will  be 
required,  and  it  is  usually  up  to  the  programmer  to  decide  for  the  im- 
plementation details.  However,  the  higher  level  design  of  a  program  always 
gives  an  indication  whether  the  implementation  requires  many  iterations  or 
a  lot  of  case  handling  etc.  Therefore  this  type  of  faults  depends  both  on  the 
design  and  the  implementation  details. 

On  the  other  hand  they  are  programmer  dependent  because  every 
programmer  has  his/her  own  weak  and  strong  points  in  developing  a 
software  design,  and  it  is  obvious  that  some  phases  of  his/her  work  will  be 
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more  prone  to  faults    than  others.  This  also  holds  true  for  both  the  design 
and  the  implementation  phases. 

3.      Taxonomical  Fault  Classification  of  a  Software  Experiment 

The  classification  of  the  known  program  faults  in  a  set  of  eight 
redundant  versions  of  a  combat  simulation  program  (Shimeall  1991a,  1991b) 
constructed  as  part  of  a  software  experiment  (Shimeall  and  Leveson  1991) 
were  analyzed.  The  fault  categories  used  in  this  taxonomy  are  from  Beiser 
(1990).  According  to  his  classification  scheme  each  fault  is  characterized  by 
the  type  of  logical  flaw  that  produced  it  (for  example  case  selection  bug, 
control  logic  bug  etc)  and  is  assigned  a  four  digit  code  number. 

The  first  digit  is  characteristic  of  the  highest  level  of  the  taxonomy 
hierarchy  (i.e..  Structural  Bug,  has  code  3xxx)  while  the  last  digit  specifies 
the  exact  category  of  the  fault  (for  example  a  Structural,  control  state  fault, 
has  code  3 1 54).  The  advantages  of  this  taxonomy  is  that  provides  an  easy 
hierarchical  and  logical  scheme,  and  the  four  digits  of  the  code  specify  the 
four  levels  of  the  used  hierarchy  (cf  Chapter  II  section  C  for  a  discussion 
of  the  highest  levels). 

The  results  of  the  fault  classification  in  the  eight  versions  are 
presented  in  Table  3.1  (detailed  results  including  fault  relative  frequency 
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histograms  for  the  eight  versions,  and  for  the  observed  to  expected  relative 
frequency  ratio  of  the  total  number  of  faults  are  presented  in  Appendix  B, 
Figures  B.l  to  B.8).  In  addition  to  the  fault  statistics,  in  the  table  we  include 
the  number  of  statements,  if  and  case  selection  statements,  and  loop 
constructs. 

4.      The  *Taxonoinical  Clusterii^"  Hypothesis 

The  testing  of  the  ^Haxonondcal  clustering^'  hypothesis  is 
performed  by  comparison  of  the  actual  results  with  a  simple  random  model 
that  assumes  that,  given  the  total  number  of  faults  in  a  program,  the 
probability  of  an  fault  occurrence  in  any  line  is  equal  to  the  ratio  of  the  total 
number  of  faults  by  the  total  number  of  lines.  The,  ratio,  is  the  expected  fault 
rate  for  the  given  program.  The  expected  number  of  faults  for  each  of  the 
categories  in  Table  3.1,  for  the  random  fault  distribution,  is  calculated  as  the 
product  of  the  fault  rate  time  the  number  of  statements  where  this  type  of 
fault  can  occur. 

Within  the  scope  of  our  working  model,  processing,  initialization 
and  algorithmic  faults  (codes  32  Ix,  322x,  323x)  occur  in  all  program 
statements  apart  from  if,  case  and  loop  constructs.  Loop  and  iteration  faults 
(code  314x)  occur  in  loop  constructs.  Control  logic,  case  selection  and 
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control  state  faults  (code  312x,  313x,  315x)  may  appear  in  if  or  case 
statements.  The  exception  handling  faults  appear  usually  either  in  control  (if, 
case)  statements  or  loop  exit  conditions.  Therefore  we  assume  further  that, 
on  the  average,  half  of  the  faults  in  loop  and  control  statements  are  on 
exception  handling  and  the  other  half  on  loop,  iteration  and  control 
respectively. 

The  random  fault  allocation  that  results  from  the  use  of  our  model, 
is  also  included  in  Table  3.1  for  the  eight  program  versions.  In  this  case  the 
"random"  version  number  has  the  subscript  r. 

In  Appendix  B  we  present  the  relative  frequency  distribution  for 
both  the  actual  number  of  faults  and  the  model  predictions.  To  test  the 
goodness  of  fit  of  the  observations  to  the  random  fault  distribution  model  we 
make  use  of  the  Chi-Square  (y^)  One-Sample-Test  (Siegel  and  Castellan 
1988).  This  test  gives  the  level  of  significance  (probability  of  occurrence  in 
fact)  that  the  y}  statistic,  which  increases  with  the  difference  between  the 
observed  and  the  expected  fault  distribution,  is  greater  than  a  certain  value. 
The  higher  the  probability  of  the  said  difference  the  more  confident  we  are 
that  the  selected  model  corresponds  to  the  actual  distribution.  However,  the 
results  of  Table  3.1  imply  that  the  probability  of  the  observed  differences 
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between  the  actual  number  of  faults  and  the  model  prediction  is  below 
0.001,  because  for  3  degrees  of  freedom  (since  there  are  four  fault  cate- 
gories) the  smallest  value  of  x^  was  19.7  in  version  VI.  Therefore  we  can 
conclude  that  the  fault  distribution  is  not  uniform.  This  implies  that  the  faults 
that  have  the  most  opportunity  of  being  committed  are  not  always  the  most 
frequent  in  a  program. 

It  is  evident  from  the  statistics  in  Table  3.1  that  control  logic,  case 
selection  and  control  state  faults  appear  at  a  rate  about  fourteen  times  the 
average  fault  ratio  in  the  program  while  processing  and  initialization  fault 
rate  is  about  30%  of  the  average.  This  leads  to  the  conclusion  that  the 
handling  of  the  control  logic  of  the  program,  at  least  in  the  CONFLICT 
experiment  (Shimeall  1991a,  1991b,  Shimeall  and  Leveson  1991)  and  for  the 
programmers  selected,  represents  a  task  of  much  greater  difficulty  than 
processing  and  initialization  in  the  same  program. 
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TABLE  3.1 
(PROGRAM  STATISTICS,  FOR  TAXONOMICAL  CLUSTERING) 
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loop 
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I 
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0 
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15 

2 

26 

Ir 
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1 
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26 

x' 
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0 

21 

3 

2 

26 

n. 
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86 
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0.73 

1.0 
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26 

x^ 

- 

- 

- 

0.73 
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16.9 

0.04 

417.7 

in 
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88 
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I 

27 

8 

3 

40 
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88 
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39 

i' 

- 

- 

- 
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18.2 
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IV 
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1 

8 

7 

3 

23 
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0.6 

1 

16 
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19 

i' 
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TABLE  3.1 
(PROGRAM  STATISTICS,  FOR  TAXONOMICAL  CLUSTERING) 


Vers- 
ion 

if  + 

case 
statem- 
ents 

loop 
const- 
ructs 

other 
statem- 
ents 

314x 
(loop 
and  ite- 
ration) 

312x, 
313x, 
315x, 

(control- 
logic, 

case  sele- 
ction, 
control 
state) 

321x, 
322x 
323x, 
(proc- 
essing 
and  in- 
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tion) 
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excep- 
tion han- 
dling 

(316x) 
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28 

6 

1 
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1.7 

29 

2.7 
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X 

- 
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1 
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18.2 

1.1 
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VI 
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1 

5 

13 

3 
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x' 
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5.      Taxonomical   Clustering   of  Failure   Regions  Compared  to   the 
Structural  Clustering  of  Failure  Regions 

In  this  section,  we  examine  whether  the  logical  clustering  of  failure 
regions  correlates  with  the  number  of  shared  dimensions  in  their  bounds.  The 
number  of  shared  dimensions  at  the  bounds  of  two  failure  regions  has  been 
used  in  the  definition  of  a  clustering  metric,  the  Jaccard  coefficient  between 
two  regions,  by  Ginn  (1991).  On  the  other  hand,  we  consider  two  failure 
regions  as  members  of  the  same  Taxonomical  cluster  if  the  associated  faults 
are  both  results  of  similar  logical  flaws.  We  classify  the  Taxonomical 
clusters  as  follows: 

•  Type  A:  Corresponds  to  loop  and  iteration  faults  (code  314x). 

•  Type  B:  Corresponds  to  control  logic,  case  selection  and  control  state 
(codes  312x,  313x,  315x)  faults. 

•  Type  C:  Corresponds  to  processing,  initialization  and  algorithmic  faults 

(codes  32  Ix,  322x,  323x). 

•  Type  D:  Corresponds  to  faults  in  exception  handling  (code  316x). 

Summary  statistics  of  this  comparison  are  presented  in  Table  3.2 
(Details  are  included  in  Appendix  D).  From  these  results  it  is  apparent  that 
the  appearance  of  shared  dimensions,  and  the  variation  of  the  Jaccard 
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coefficient,  on  failure  region  bounds  does  not  depend  on  the  taxonomical 
clustering  as  analyzed  in  subsection  b,  where  the  clustering  criterion  between 
different  failure  regions  has  been  whether  they  correspond  to  faults  resulting 
from  the  same  logical  flaw. 


TABLE  3.2 
AVERAGE  AND  STANDARD  DEVIATION  OF  THE  JACCARD 
COEFHCIENT  BETWEEN  TAXONOMICALLY  CORRELATED 

FAILURE  REGIONS 


VER- 
SION 

Fmlaie 
re(>ion  pair 
of  type  A 

Failure  region 
pair  of  type  B 

Failure  region 
pair  of  type  C 

Failure  region 
pair  of  type  D 

Uncoirciated 
failure  region  pair 

I 

- 

0.051±0.115 

0.04910.134 

0.25' 

0.04210.108 

n 

- 

0.050±0.149 

0.133 

- 

0.02910.066 

m 

- 

0.076±0.135 

0.05410.107 

0 

0.05510.115 

rv 

- 

0.032±0.061 

0.05010.118 

0.077 

0.03610.092 

V 

- 

0.01910.085 

0.01710.034 

- 

0.00810.035 

V. 

- 

0.073+0.230 

0.05910.118 

- 

0.01810.039 

vn 

- 

0.074±0.174 

0.03110.075 

0.11110.19 

0.01810.065 

vin 

- 

0.123±0.429 

- 

0.13410.181 

0.11710.227 

In  Table  3.3  we  present  the  same  statistics  as  in  Table  3.2  with 
the  difference  that,  in  this  case,  we  use  the  average  non-zero  Jaccard 
coefficient  between  taxonomically  correlated  failure  region.  This  indicates 


In  entries  without  Standard  Deviation  there  is  only  one  value. 
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whether  the  taxonomical  clustering  of  failure  regions  correlates  with  the 
number  of  shared  dimensions  in  their  bounds,  when  these  shared  dimensions 


exist. 


TABLE  3.3 
AVERAGE  AND  STANDARD  DEVIATION  OF  THE  NON  ZERO 
JACCARD  COEFHCIENT  BETWEEN  TAXONOMICALLY 
CORRELATED  FAILURE  REGIONS 


VER- 
SION 

Failiuc 
region  pair 
of  type  A 

Failure  region 
pair  of  type  B 

Failure  region 
pair  of  type  C 

Failure  region 
pair  of  type  D 

Unconelated 
failure  region  pair 

I 

- 

0.124±0.150 

0.247±0.361 

0.250 

0.164±0.159 

n 

- 

0.393±0.196 

0.133 

- 

0.128+0.09 

m 

- 

0.171±0.139 

0.139±0.134 

- 

0.159±0.143 

IV 

- 

0.129±0.048 

0.156±0.167 

0.077 

0.131+0.136 

V 

- 

0.315±0.172 

0.067 

- 

0.115±0.089 

VI 

- 

0.73 

0.191±0.147 

- 

0.086+0.026 

vn 

- 

0.434±0.147 

0.133±0.109 

0.333 

0.173±0.110 

vm 

- 

0.469±0.222 

- 

0.283±0.117 

0.387±0.258 

The  results  in  Table  3.3  provide  us  with  evidence  in  favor  of  the 
intuitively  obvious  hypothesis  that  the  structural  clustering  of  failure  regions 
does  not  depend  on  the  taxonomical  clustering.  This  result  can  be  attributed 
to  the  fact  that  structural  clustering  of  failure  regions,  depends  strongly  on 
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the  control  and  data  flow  structure  of  the  program,  with  respect  to  the 
corresponding  fauh  location.  On  the  other  hand  the  taxonomical  clustering 
dependence  is  restricted  to  the  type  of  statement  at  the  fault  location. 

6.  The  Case  of  Functionally  Related  Faults 

The  hypothesis,  that  logically-related  faults  located  in  regions  of 
programs  that  compute  the  same  part  of  the  application  may  lead  to  some 
type  of  clustering,  is  based  on  the  intuitively  obvious  assumption  that  some 
parts  of  a  problem  may  be  more  difficult  to  handle  or  more  "error  prone" 
than  others  (Brilliant  et  al  1990).  We  shall  call  this  type  of  faults /ii/tcft'^- 
nally  related. 

7.  Functional  Fault  Classification  of  a  Software  Experiment 

The  distribution  of  the  known  program  faults  in  the  eight  versions 
of  the  combat  simulation  program  (Shimeall  1991,  1991b)  to  the  program 
modules  implementing  different  functional  requirements  of  the  specification 
was  analyzed.  We  classified  the  ftmctional  clusters,  according  to  the 
CONFLICT  Specification  (Shimeall    1991b)  as  follows: 


•    Type  I:  Positioning  and  Movement 
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•  Type  H:  Observation 

•  Type  in.  Attrition 

•  Type  TV.  Communication 

•  Type  V:  Environment 

•  Type  VI.  Restoration 

•  Type  O.  Others  (Includes  the  main  procedure  of  CONFLICT  and  some 
procedures  for  initialization  and  output  format) 

8.      The  Functional  Clustering   Hypothesis 

The  null  hypothesis  that  fault  distribution  is  uniform  over  the 
program  locations  was  used  in  this  analysis  to  test  the  clustering  hypothesis. 
The  number  of  expected  faults  for  each  functional  requirement  of  the 
specification  was  set  equal  to  the  total  number  of  lines  of  routines  im- 
plementing the  requirement  times  the  fault  rate  (as  in  subsection  lb)  for  the 
program. 

The  results  of  this  analysis  are  presented  in  Table  3.4.,  while 
detailed  analysis   is   included  in  Appendix  D. 

To  test  the  goodness  of  fit  of  the  observations  to  the  random  fault 
distribution  model  we  make  use  of  the  Chi-Square  (y^)  One-Sample-Test 
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(Siegel  and  Castellan  1988),  as  in  subsection  4.  In  this  case  there  are  seven 
categories  of  faults,  therefore  the  degrees  of  freedom  are  six. 

Apart  from  versions  IV,  VII  and  VIII,  the  confidence  level  for  the 
null  hypothesis  is  varying  from  1%  to  31%.  In  versions  IV,  VII  and  VIII  this 
confidence  drops  below  0.1%.  This  result,  however,  may  be  attributed  to  the 
low  (less  than  five)  expected  number  of  type  I,  II,  III,  V,  VI  and  O  faults  for 
version  IV  and  the  low  expected  number  of  type  I,  V,  VI  faults  for  version 
VIII.  In  version  VII  the  result  may  be  attributed  to  the  low,  3.1,  number  of 
expected  fault  at  column  IV  compared  to  the  10.5  observed. 

The  above  discussion  can  be  verified  in  case  we  consider  only  two 
fault  categories  for  version  IV,  Type  IV  with  7  observed  and  6.7  expected 
faults,  and  all  others  with  16  observed  and  16.3  expected  faults.  This  will 
result  in  a  x"^  equal  to  0.019  and  a  corresponding  confidence  level,  one 
degree  of  freedom  this  time,  greater  than  90%. 

In  the  case  of  version  VIII,  we  may  consider  the  fault  categories 
II,  III,  IV,  O  and  all  others,  with  four  degrees  of  freedom,  and  the  y^  will  be 
equal  to  14.85,  which  gives  a  confidence  level  of  1%. 

The  confidence  level  increases  accordingly  with  all  the  remaining 
versions  if  we  group  together  all  categories  with  expectation  value  less  than 
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5  as  suggested  by  Siegel  and  Castellan  (1988).   Under  the  circumstances,  we 
cannot  either  accept  or  reject  the  null  hypothesis  of  uniform  distribution  of 


faults. 


An  alternate  hypothesis,  that  the  faults  are  uniformly  distributed 


between  type  IV  (Communication),  type  III+II  (Attrition  and  Observation) 
and  all  other  types  is  tested  in  Table  3.5.  The  alternate  hypothesis  assumes 
that  some  type  of  functional  clustering  exists,  because  2/3  of  the  faults  are 
clustered  in  the  Communication,  Attrition  and  Observation  modules,  which, 
on  the  average,  constitute  the  50%  of  the  total  lines  of  code. 


TABLE  3.4 
(PROGRAM  STATISTICS,  FOR  FUNCTIONAL  CLUSTERING) 


VenioD 

Type  I 

Type 

n 

Type 

m 

Type 

rv 

Type 
V 

Type 
VI 

Type 
O 

TO- 
TAL 

Confidence 
Level 

I 

1 

3 

4 

3 

4 

3 

8 

26 

- 

K 

1.77 

2.65 

3.92 

7.49 

1.58 

2.38 

6.2 

26 

- 

2 

X 

0.33 

0.05 

0.002 

2.7 

3.71 

0.16 

0.52 

7.5 

28.4% 

n 

3 

5 

0 

14 

3 

1 

0 

26 

- 

n^ 

4.10 

3.83 

4.85 

6.23 

2.38 

1.5 

3.11 

26 

- 

2 
1 

0.295 

0.008 

4.85 

9.7 

0.16 

0.17 

3.11 

18.3 

1% 
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TABLE  3.4  (PROGRAM  STATISTICS,  FOR  FUNCTIONAL  CLUSTER- 
ING) 


Venion 

Type  I 

Typen 

Typeffl 

Type  IV 

TypeV 

Type 
VI 

Type 

o 

TO- 
TAL 

Confidence 
Level 

m 

2 

3 

7 

13.5 

5 

2 

7.5 

40 

- 

fflr 

3.13 

4.9 

4.37 

13.57 

2.83 

0.8 

10.4 

40 

- 

2 

X 

0.41 

0.74 

1.6 

0.00 

1.66 

1.8 

0.81 

7.0 

31% 

IV 

0 

11.5 

2.5 

7 

1 

1 

0 

23 

IV. 

2.4 

3.07 

3.7 

6.7 

1.7 

2.3 

3.1 

23 

- 

2 

X 

2.4 

23.15 

0.39 

0.013 

0.29 

0.04 

3.1 

29.4 

<0.1% 

V 

0 

6 

2 

15.5 

1.5 

2 

13 

40 

- 

V. 

2.6 

7.9 

7.3 

9.0 

2.3 

1.5 

9.4 

40 

- 

x' 

2.6 

0.60 

3.85 

4.69 

0.28 

0.17 

1.38 

13.5 

5% 

VI 

0 

3 

5 

7.5 

0 

0 

4.5 

20 

- 

VI. 

0.87 

2.9 

1.8 

8.1 

1.6 

1.03 

3.7 

20 

- 

2 

X 

0.87 

0.00 

5.7 

0.04 

1.6 

1.03 

0.17 

9.4 

25% 

vn 

0 

7 

5 

10.5 

2 

2 

4.5 

31 

- 

vn. 

5.0 

4.6 

3.8 

3.1 

2.4 

2.0 

10.4 

31 

- 

2 

X 

5.0 

1.25 

0.38 

17.7 

0.07 

0 

3.35 

27.8 

<0.1% 

vra 

4 

11 

3 

15 

7 

1 

0 

41 

- 

vm. 

4.2 

5.9 

6.4 

10.8 

3.3 

2.7 

7.8 

41 

- 

x' 

0.009 

4.41 

1.8 

1.63 

4.15 

1.1 

7.8 

21.0 

0.1% 

TOTAL 

10 

49.5 

28.5 

86 

23.5 

12 

37.5 

247 

- 

TOTALr 

24.1 

35.6 

36.1 

65 

18.1 

14.23 

62.8 

247 

- 

x' 

22.7 

5.43 

1.6 

6.78 

1.61 

0.35 

10.2 

48.6 

<0.01% 
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TABLE  3.5  (ALTERNATE  HYPOTHESIS  TEST) 


Version 

Type  rv 

Type  n+ffl 

Type 
(KI+V+VI 

TOTAL 

Confidence 
Level 

I 

3 

7 

16 

26 

- 

K 

8.67 

8.67 

8.67 

26 

- 

l' 

3.71 

0.32 

6.2 

10.23 

0.6% 

n 

14 

5 

7 

26 

- 

Hr 

8.67 

8.67 

8.67 

26 

- 

2 

X 

3.28 

1.55 

0.32 

5.15 

7.6% 

m 

13.5 

10 

16.5 

40 

- 

nir 

13.33 

13.33 

13.33 

40 

- 

2 

X 

0.00 

0.833 

0.752 

1.59 

92% 

rv 

7 

14 

2 

23 

- 

1       ^' 

7.67 

7.67 

7.67 

23 

- 

i' 

0.06 

5.22 

4.19 

9.48 

0.9% 

V 

15.5 

8 

16.5 

40 

- 

V, 

13.33 

13.33 

13.33 

40 

- 

2 

X 

0.35 

2.13 

0.75 

3.23 

20% 

VI 

7.5 

8 

4.5 

20 

- 

VI. 

6.67 

6.67 

6.67 

20 

- 

l' 

0.10 

0.27 

0.71 

1.08 

60% 

vn 

10.5 

12 

8.5 

31 

- 

vn. 

10.33 

10.33 

10.33 

31 

- 

2 

X 

0.00 

0.27 

0.33 

0.60 

75% 
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TABLE  3.5  (ALTERNATE  HYPOTHESIS  TEST) 


Version 

Type  rV 

Type  n+m 

Type 
O+I+V+VI 

TOTAL 

Confidence  Level 

vra 

15 

14 

12 

41 

- 

vra. 

13.67 

13.67 

13.67 

41 

- 

x' 

0.13 

0.01 

0.20 

0.34 

84.3% 

TOTAL 

86 

78 

83 

247 

- 

TOTALr 

82.33 

82.33 

82.33 

247 

- 

X^ 

0.16 

0.23 

0.01 

0.40 

82% 

The  confidence  levels  of  the  chi-square  test  range  from  84.3%  to 
0.6%.  Although  we  cannot  accept  or  reject  the  alternate  hypothesis,  we 
notice  that  the  confidence  levels  for  acceptance  are  in  general  higher  that 
those  of  the  null  hypothesis.  The  results  for  versions  III,  VI,  VII  and  VIII  are 
in  favor  of  the  alternate  hypothesis,  which  implies  some  type  of  functional 
clustering.  The  results  for  versions  II,  IV,  V  are  not  in  favor  of  the  alternate 
hypothesis  because  the  functional  clustering  tendency  is  stronger  than  the 
assumed  by  the  alternate  hypothesis,  and  only  version  I  implies  that  there  is 
not  any  clustering  tendency  favoring  Attrition  Communication  and  Observa- 


tion. 
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Therefore  it  is  reasonable  to  conclude  from  the  data  at  hand  that  there  is 
indeed  some,  not  very  strong,  tendency  of  the  failure  regions  to  cluster  in 
some  groups  of  ftinctional  program  modules  more  than  in  others. 

9.      Clustering  of  Failure  Regions  of  Functionally  Related  Faults  Com- 
pared to  the  Structural  Clustering  of  Failure  Regions 

In  this  section,  we  examine  whether  the  functional  clustering  of 
failure  regions  correlates  with  the  number  of  shared  dimensions  in  their 
bounds.  In  this  case  we  consider  two  failure  regions  as  members  of  the 
same  functional  cluster  if  the  associated  faults  appear  in  regions  of  the  | 
program  that  compute  the  same  part  of  the  application.  Summary  statistics 
of  this  comparison  are  presented  in  Table  3.6  and  3.7  (the  latter  corresponds 
to  the  non  zero  Jaccard  coefficient  case)  while  details  are  included  in 
Appendix  D. 

From  the  results  of  Table  3.6  it  is  not  possible  to  conclude  that  the 
appearance  of  shared  dimensions,  and  the  variation  of  the  Jaccard  coefficient, 
on  failure  region  bounds  does  or  does  not  depend  on  the  functional 
clustering,  as  defined  in  subsection  8,  because  the  calculated  standard 
deviations  exceed  the  averages  due  to  the  abundance  of  zero  data  values. 
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However,  from  Table  3.7,  we  can  see  that  the  average  non  zero 
Jaccard  coefficient  is  systematically  higher  for  functionally  correlated  pairs 
that  for  uncorrelated.  This  implies  that,  given  the  structural  clustering  of  two 
failure  regions,  the  clustering  metric  is  higher  for  functionally  correlated 
ones.  This  result  can  be  attributed  to  the  fact  that  structural  clustering  of 
failure  regions  depends  on  the  control  and  data  flow  structure  of  the 
program,  with  respect  to  the  corresponding  fault  location. 

It  is  expected  that  in  a  reasonably  well  structured  program,  faults 
on  the  same  group  of  functional  modules  will  share  a  common  control  emd 
data  flow  path  segment  more  often  than  faults  on  different  groups. 
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TABLE  3.6 

AVERAGE  AND  STANDARD  DEVIATION  OF  THE  JACCARD 

COEFHCIENT  BETWEEN  FUNCTIONALLY  CORRELATED 

FAILURE  REGIONS^ 


VERSION 

Failme 
Region 
Pair  of 
Type  I 

Failure 
Region 
Pair  of 
Typen 

Failure 
Region 
Pair  of 
Typem 

Failmc 
Region 
Pair  of 
TypelV 

Fdlmc 
Region 
Pair  of 
TypeV 

Failure 
Region 
Pair  of 
Type  VI 

Failare 
Region 
Pair  of 
TypeO 

Failure 
Region 

Pair 
Uncor- 
related 

1 

- 

0.025 
±0.04 

0.140 
±0.006 

0.042 

0.240 
±0.49 

0.125 
±0.21 

0.099 
±0.19 

0.0284 
±0.081 

n 

0±0 

0.091 
±0.24 

- 

0.076 
±0.184 

0.111 
±0.192 

- 

- 

0.0181 

±0.073 

m 

0.250 

0±0 

0.223 
±0.312 

0.097 
±0.137 

0.148 
±0.091 

0 

0.096 
±0.14 

0.046 
±0.095 

IV 

- 

0.0374 
±0.064 

0.056 
±0.096 

0.0823 

±0.235 

- 

- 

- 

0.019 
±0.053 

V 

- 

0±0 

0.714 

0.0227 
±0.086 

0 

0 

0.005 
±0.03 

0.0087 
±0.051 

VI 

- 

0.336 
±0.043 

0.1142 
±0.220 

0.075 
±0.127 

- 

- 

0.0643 
±0.16 

0.0184 
±0.071 

vn 

- 

0.0263 
±0.058 

0.2922 
±0.239 

0.131 
±0.220 

0.333 

0 

0.117 
±0.19 

0.0192 
±0.071 

vm 

0.125 
±0.25 

0.333 
±0.315 

0.75 

0.102 
±0.206 

0.187 
±0.285 

- 

- 

0.069 
±0.180 

In  entries  without  Standard  Deviation  there  is  only  one  value. 
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TABLE  3.7 
AVERAGE  AND  STANDARD  DEVIATION  OF  THE  NON  ZERO 
JACCARD  COEFFICIENT  BETWEEN  RINCTIONALLY 
CORRELATED  FAILURE  REGIONS' 


VERSION 

Failme 
Region 
Pair  of 
Type  I 

Failme 
Region 
Pair  of 
Typen 

Faiime 
Region 
Pair  of 
Typem 

Failme 
Region 
Pair  of 
TypelV 

Failme 
Region 
Pair  of 
TypeV 

Failme 

Region 

Pair  of 

Type  VI 

Failme 
Region 
Pair  of 
TypeO 

Failme 
Region 

Pair 
Uncop- 
related 

1 

- 

0.074 

0.1 40± 
0.0060 

0.042 

0.7921 

0.3501 

0.430 

± 
0.110 

0.121 
10.131 

n 

- 

0.636 

- 

0.407± 
0.216 

0.333 

- 

- 

0.2061 
0.152 

m 

0.250 

- 

0.586± 
0.189 

0.2441 
0.168 

0.1481 
0.091 

- 

0.240 

1 
0.149 

0.1451 
0.116 

IV 

- 

0.127± 
0.053 

0.167 

0.70 

- 

- 

- 

0.1031 
0.080 

V 

- 

- 

- 

0.2981 
0.160 

- 

- 

0.170 

1 
0.029 

0.1701 
0.177 

VI 

- 

0.336± 
0.043 

0.1 90± 
0.265 

0.1861 
0.156 

- 

- 

0.50 

0.1001 
0.070 

vn 

- 

0.106+ 
0.073 

0.2922 
±0.239 

0.4761 
0.088 

0.333 

- 

0.389 

+ 
0.096 

0.1961 
0.130 

vm 

0.500 

0.438± 
0.285 

0.75 

0.4421 
0.182 

0.4271 
0.269 

- 

- 

0.3941 
0.237 

In  entries  without  Standard  Deviation  there  is  only' one  value.  Crossed  out 
entries,  contain  Average  and  Standard  deviation  of  only  two  data  values 
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C.     CONCLUSIONS  ON  CHAPTER  ID 

In  this  chapter  the  failure  region  analysis  on  the  results  of  the 
CONFLICT  experiment  identifies  the  logical  relation  of  the  faults  responsible 
for  the  observed  failure  regions. 

The  testing  of  the  taxonomical  clustering  hypothesis,  section  3.3  to  3.5, 
implies  the  observed  fault  distribution  is  not  uniform.  The  control  logic,  case 
selection,  and  control  state  faults  appear  at  a  rate  about  fourteen  times  the 
average  fault  ratio  in  the  program  while  processing  and  initialization  fault 
rate  is  about  30%  of  the  average.  Therefore  control  logic  faults  and 
corresponding  failure  regions  actually  exhibit  taxonomical  clustering 
behavior.  This,  however,  does  not  correlate  with  the  structural  clustering  of 
failure  regions  observed  by  Ginn  (1991)  on  the  same  set  of  data.  This  can 
be  justified  by  the  fact  that  the  latter  depends  strongly  on  the  control  and 
data  flow  structure  of  the  program,  with  respect  to  the  corresponding  fault 
location,  while  the  former  depends  on  the  type  of  statement  at  the  fault 
location. 

The  results  of  subsections  3.6  to  3.8  show  that  about  2/3  of  the  faults, 
on  the  average,  appear  in  the  functional  modules  of  Attrition,  Communica- 
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tion,  and  Observation  which  include  approximately  50%  of  the  program 
lines.  This  implies  some  mild  tendency  of  the  failure  regions  for  functional 
clustering,  since  some  functional  modules  are  indeed  more  fault-prone  than 
others. 

From  subsection  3.9  we  can  see,  comparing  our  results  with  Ginn's 
(1991),  that  for  all  pairs  of  structurally  correlated  failure  regions,  the 
clustering  metric  is,  on  the  average,  higher  for  functionally  correlated  ones. 
This  is  justified  by  fact  that,  in  a  reasonably  well  structured  program,  faults 
on  the  same  group  of  functional  modules  will  share  a  common  control  and 
data  flow  path  segment  more  often  than  faults  on  different  groups,  therefore 
functional  and  structural  correlation  cannot  be  independent. 
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IV.  CONCLUSIONS  AND  SUGGESTIONS  FOR  FURTHER  RE- 
SEARCH 

It  has  been  conjectured  in  the  past  that  fauh  occurrences  tend  to 
converge  on  program  locations,  which  implies  that  the  revealing  of  a  fault 
might  indicate  the  existence  of  others  in  close  proximity  of  location. 
However,  the  evidence  so  far  has  been  mostly  anecdotal.  This  thesis,  together 
with  Ginn's  (1991)  have  been  of  the  first  to  analyze  the  relationships  between 
specific  faults  using  structural  (Ginn  1991),  taxonomical  and  functional  (this 
thesis)  criteria.  The  results  of  both  support  the  hypothesis  of  fault  clustering 
and  suggest  methods  for  the  exploitation  of  them  in  software  testing.  This 
chapter  summarizes  these  results,  in  conjunction  with  previous  work,  and 
points  towards  the  research  questions  which  are  open  to  further  investigation. 

A.     CONCLUSIONS 

This  thesis,  being  a  sequel  to  previous  work  by  Ginn  (1990),  offers 
strong  evidence  that  failure  regions  tend  to  form  clusters,  not  only  when 
structural  criteria  are  used,  but  taxonomical  and  functional  as  well. 
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The  clustering  criteria  used  in  this  thesis  were  imposed  externally,  since 
both  the  fault  taxonomy  and  the  functional  classification  of  the  faults  have 
been  independent  from  the  experimental  data  structure.  Therefore,  the 
observed  clustering  tendency  of  failure  regions  can  be  characterized  as  global 
as  opposed  to  the  local  clustering  tendency  explored  b  Ginn  (1990).  In  local 
clustering  the  criteria  are  strongly  dependent  upon  the  structure  of  the  data 
at  hand  (Jain  and  Dubes  1988).  Therefore  the  Jaccard  coefficients  used  by 
Ginn  readily  fall  into  this  category. 

The  taxonomical  clustering  behavior  suggests  that  parts  of  the  program 
prone  to  control  logic,  case  selection  and  control  state  faults,  must  be  the 
focus  of  the  testing  effort.  This  implies  that  these  parts  of  a  program  must 
also  be  thoroughly  and  extensively  documented  in  order  to  facilitate  this 
focus  of  effort.  The  CONFLICT  experiment  (Shimeall  1991a,  b)  data 
analysis  suggests  that  decision  points  and  program  control  flow  have  a  higher 
probability  of  fault  occurrence  than  other  locations  of  the  code.  A  good 
testing  or  documentation  method,  such  as  decision  tables  etc.,  is  expected  to 
reduce  substantially  the  rate  of  this  type  of  faults. 

The  usefulness  of  the  functional  clustering  behavior  is  that  known  faults 
in  a  program  imply  that  the  probability  of  more,  undiscovered,  faults  in  the 


45 


functional  module  is  higher  than  the  in  rest  of  the  code.  This  provides  expe- 
rimental evidence  in  support  of  the  anecdotal  conjecture  that  faults  tend  to 
attract  other  faults  (Myers  1979).  It  also  suggests  that  the  distribution  of 
faults  during  the  first  tests  indicates  the  most  fault-prone  functional  modules, 
which  should  be  singled  out  for  additional  testing. 

More  often  than  not  the  testing  effort  is  exponential,  or  of  higher 
complexity,  in  the  length  of  code.  The  ability  to  point  out  the  most  fault- 
prone  modules  or  constructs,  under  the  experimentally-verified  assumption 
of  functional  and  taxonomical  clustering,  represents  a  substantial  reduction 
to  the  required  amount  of  testing. 

The  nature  of  the  cluster  formation,  and  the  correlation  to  Ginns 
structural  clustering,  for  the  two  criteria  used  was  markedly  different.  The 
taxonomical  classification  tended  to  demonstrate  a  clustering  of  type  C  faults 
(and  failure  regions)  in  numbers  one  order  of  magnitude  higher  than  the 
expected  when  a  uniform  fault  per  line  of  code  distribution  was  assumed. 

In  order  to  compare  the  taxonomical  clustering  of  failure  regions  to  the 
structural  clustering  of  the  same  regions,  we  calculated  both  the  average 
Jaccard  coefficient  and  the  average  non-zero  Jaccard  coefficient  (using  Ginn's 
results)  for  every  type  of  possible  taxonomical  clustering  and  every  version 
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of  the  program.  The  results  in  Tables  3.2  and  3.3,  in  Chapter  III,  suggest  that 
the  structural  clustering  of  failure  regions  does  not  depend  on  taxonomical 
clustering.  This  is  attributed  to  the  fact  that  structural  clustering  depends 
strongly  on  the  control  and  data  flow  structure  of  a  program  with  respect  to  the 
corresponding  fault  locations  while  the  taxonomical  clustering  depends  mainly  on 
the  type  of  program  statement  at  the  said  locations. 

The  functional  criterion  revealed  a  tendency  of  faults  to  concentrate  on 
certain  fimctional  groups  of  modules  (In  the  CONFLICT  case  in  the  Communication, 
Observation  and  Attrition  groups).  A  quite  interesting  result,  in  this  analysis, 
has  been  the  small  scale  clustering  exhibited  by  the  faults,  which  tended  to  occur 
in  high  numbers  within  certain  procedures  (93)  while  the  majority  of  the  examined 
procedures  (446  total,  in  all  eight  versions  of  CONFLICT)  were  faultless  (cf 
Appendix  D  for  a  detailed  functional  fault  type  distribution).  Similar  behavior 
of  faults  has  been  reported  by  Myers  (1979)  but  no  explanation  was  cited.  The 
small-scale  clustering  suggests  that  some  procedures  are  more  complicated, 
therefore  more  fault-prone,  than  others.  In  the  eight  versions  of  the  CONFLICT 
experiment,  the  average  procedure  was  30±20  lines  of  code  in  length  while  the 
average  length  of  procedures  with  at  least  one  fault  has  been  42.5±33  lines  and 
with  two  or  more  faults  55±45  lines.  This  result  provides  evidence  in  support  of 
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the  above-mentioned  argument  about  procedure  complexity  and  fault 
clustering  correlation,  despite  the  inaccuracies  introduced  by  the  fact  that 
only  faults  with  a  well-defined  location  were  considered. 

Unlike  the  taxonomical-to-structural  clustering  lack  of  correlation,  in 
testing  the  functional-to-structural  clustering  correlation  in  the  same  as  above 
way,  it  was  found  that  the  average  non-zero  Jaccard  coefficient  is  systemati- 
cally higher  for  functionally  correlated  pairs  than  for  uncorrelated.  This  mild 
correlation  is  explained  by  the  fact  that  in  a  reasonably  well-structured 
program,  faults  of  the  same  group  of  functional  modules  will  share  a 
common  control  and  data  flow  path,  will  be  structurally  correlated,  more 
often  than  faults  on  different  groups. 

The  use  of  the  number  of  lines  of  code  (excluding  comments  but  not 
variable  declarations)  in  our  analysis  instead  of  the  Halstead  length  is  due 
to  the  ease  of  use  of  this  metric  as  well  as  to  unpublished  calculations. 
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B.      COMPARISON  OF  RESULTS  TO  PREVIOUS  WORK 

The  results  of  this  thesis  generally  support  the  findings  of  previous 
researchers  in  the  area  of  relationships  between  faults.  This  agreement 
suggests  that  the  relationships  may  not  be  specific  to  the  CONFLICT 
software  experiment  but  have  a  more  general  vahdity  in  large  software 
applications. 

Myers  (1979)  postulates  that  functional  clustering  exists  but  provides 
no  further  evidence  or  explanation  of  the  basis  for  it.  This  thesis,  together 
with  the  companion  work  of  Ginn  (1990)  makes  a  step  toward  identifying 
the  specific  behavior  of  faults  that  result  in  failure  region  clustering. 

The  eminent  role  of  control  logic,  case  selection  and  control  state  (type 
C)  faults  in  program  testing  has  been  always  emphasized  (Beiser  1990, 
Myers  1979),  therefore  the  importance  of  taxonomical  clustering  cannot  be 
easily  overlooked. 

Briliant  et  al  (1990),  analyzing  the  faults  in  a  27-version  software 
experiment,  conclude  that  the  faults  across  independent  versions  not  only  are 
not  independent  but  the  interdependence  is,  in  many  cases,  more  pronounced 
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among  logically  related  (which  includes  taxonomically  and  functionally 
related)  faults. 

Further  study  of  the  fault  relations  on  the  same  version  or  across 
versions  is  required  for  establishment  of  the  interaction  mechanisms. 

C.      SUGGESTIONS  FOR  FURTHER  RESEARCH 

While  the  results  of  this  work  are  promising,  the  experimental 
population  was  small  and  narrowly  focused.  Additionally,  the  programs  were 
written  by  students.  Both  the  method  and  the  results  should  be  validated 
using  a  broad  range  of  professionally-produced  applications. 

One  weakness  of  the  method  used  in  this  thesis  is  that  in  testing  the 
taxonomical  clustering  hypothesis,  a  small  number  of  faults  (about  10%) 
cannot  be  classified  and  therefore  they  are  not  included  in  the  analysis.  This 
is  expected  to  introduce  some  minor  inaccuracies  in  the  results. 

Another  weakness  is  the  use  of  seven  functional  groups  of  modules  in 
the  functional  clustering  analysis.  Movement,  Observation,  Attrition, 
Communication,  Environment,  Restoration  and  Others.  This  coarse  functional 
decomposition,  imposed  by  the  necessity  of  a  common  base  for  comparison 
for  all  eight  versions  of  CONFLICT,  blurs  the  small  scale  clustering.  In 
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addition,  the  number  of  lines  per  module  is  quite  a  weak  metric  for  a 
procedure  complexity  and  cannot  be  used  effectively  as  an  oracle  to  single 
out  the  most  fault-prone  modules.  However,  this  simple  metric  indicates  that 
it  is  possible  to  use  some  normal  metric  as  a  predictor  of  the  complexity  of 
a  program  unit.  Further  research  is  required  to  develop  metrics  that  can  serve 
as  oracles  for  fault  prone  modules. 

Finally,  the  Jaccard  coefficient,  based  on  the  number  of  shared 
dimensions  bounding  two  failure  regions,  is  not  a  very  efficient  measure  of 
structural  clustering.  A  better  way  to  estabHsh  a  metric  of  two-fault 
correlation  might  be  the  use  of  the  number  of  paths  through  both  their 
respective  locations.  This  method  was  not  used  in  this  thesis  since  we  have 
been  focused  on  taxonomical  and  functional  rather  than  structural  clustering. 
However,  once  certain  ambiguities,  such  as  distributed  faults,  or  faults 
without  a  specific  location,  such  as  a  missing  function  etc.,  are  resolved  the 
proposed  metric  will  directly  translate  the  relationship  between  two  failure 
regions  into  two  sets  of  code  locations. 
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APPENDIX  A:  NOTATION  AND  SYMBOLS 

•  Dj,   Partition  of  the  input  domain,  by  path  j 

•  P(j):  Boolean  expression  equal  to  the  conjunction  of  all  branch 
predicate  constraints  along  path  j.  Therefore  all  elements  of  Dj  satisfy 

PU). 

•  Bound(G):   The  boolean  expression  which  defines  failure  region  G. 

•  J^   Symbol  for  exclusive   or  of  boolean  expressions. 

•  SjiSj2,.Sjj(D%-  The  mapping  of  a  subset,  D',  of  the  input  domain  to  the 
output  range,  by  a  sequence,  SjiSj2..Sjn,  of  program  statements  (this 
sequence  is  called  the  path  product),  where  j  is  a  path.  Condition 
evaluations  are  retained  in  the  path  products  for  consistency.  In  fact,  the 
path  products  of  the  two  segments  starting  from  a  decision  point  (such 
as  an  if  C  then  else  construct  or  a  loop  exit  condition  etc)  are  preceded 
with  the  condition  C  and  condition  -iC  respectively,  depending  upon 
which  part  of  the  decision  (if  or  else)  results  in  their  execution. 

•  SjjSjJ  SjjSjj^j,Sjj^„  )*"Sj^  •  A  sequence  of  program  statements,  when  a 
subset  of  them,  SjjSjj+,..Sjj+n„  is  ttie  body  of  a  loop. 

•  SjjSj/  SjjSp^j.Sj^^„  y'-'-^j^  '  A  sequence  of  program  statements,  when 
a  subset  of  them,  SjjSj4+i..Sj4+n„  is  the  body  of  a  loop,  which  executes  at 
least  once. 

•  S(Gi,Gj):  The  simple  matching  coefficient  for  failure  regions  Gj  and  Gj. 


52 


5(GpGp  =  ^^^ 


where  a^o  stands  for  the  non-bounding  dimensions  of  both  failure 
regions  a^  for  the  composite  dimensions  of  Gj  and  Gj  and  a  for  the 
number  of  all  input  dimensions. 


•  J(Gi,Gj  ):  The  Jaccard  coefficient  for  failure  regions  Gi  and  Gj. 


flu 


fl-floo 

where  aoo  stands  for  the  non-bounding  dimensions  of  both  failure 
regions  an  for  the  composite  dimensions  of  Gj  and  Gj  and  a  for  all 
input  dimensions. 

•  A— >B  or,  equivalently,  B^A  :  Logical  imphcation,  A  implies  B  (  or 
-lAvB). 

•  Identically  participating  dimensions:  The  dimensions  that  appear  in 
both  boundaries  of  a  pair  of  failure  regions  in  exactly  the  same  way. 

•  Coincide ntally  Participating  Dimensions:  The  dimensions  that  appear 
in  both  boundaries  of  a  pair  of  failure  regions  but  not  in  an  identical 
way. 

•  Nonbounding  Dimensions:  The  dimensions  that,  for  a  pair  of  failure 
regions,  do  not  appear  in  the  boundaries  of  both  regions.  The  boun- 
daries these  dimensions  place  on  these  failure  regions  are  no  more 
restrictive  than  their  entire  range  of  values. 

•  Composite  Dimensions:  Collective  name  for  both  Identical  and 
Coincidental  dimensions 
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Chi-Square  Statistic  (x^).  Statistic  used  to  test  whether  a  significant 
difference  exists  between  an  observed  and  an  expected  number  of 
objects^  (the  expected  number  of  objects  results  from  an  assumed 
distribution  of  objects  into  categories).  The  greater  the  chi-square 
statistic  the  lower  the  confidence  that  the  sample  data  follow  the 
assumed  distribution.  If  there  are  k  categories  of  objects,  Oj  is  the 
number  of  observed  and  Ej  the  number  of  expected  objects  in  category 
j,  then  the  statistic  has  k-1  degrees  of  freedom  and  is  equal  to: 


*  ^  IT 


Siegel  and  Castellan  1988 
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APPENDIX  B:  TAXONOMICAL  FAULT  TYPE  DISTRIBUTION 

In  this  Appendix  we  present  histograms  (Figures  Bl  to  B8,  in  Hght 
grey)  of  the  relative  frequency  of  occurrence  of  fault  types  in  the  eight 
versions  of  CONFLICT  (Shimeall  1991).  The  fault  types  in  the  histograms 
are  LOOP,  CONTROL,  PROCESS  and  EXCEPTION  HANDLING.  They 
correspond  to  the  faults  of  type  A,  B,  C  and  D  of  Chapter  HI  (In  fact  A,  B, 
C,  D  are  just  abbreviations). 

The  fault  type  frequency  histograms  are  compared  with  the  expected 
frequency  histograms  of  a  simple  model  which  assumes  unifomi  distribution 
of  faults  (histograms  in  dark  grey).  In  figure  B9  we  present  an  overall 
comparison  of  the  observed  to  expected  number  of  faults  ratio,  for  all  eight 
versions.  There  is  a  very  sharp  peak  of  the  ratio  distributions  in  the 
CONTROL  type  of  faults,  which  impHes  that  this  type  of  fault  has  a 
frequency  of  occurrence  much  greater  than  the  expected  by  the  assumed 
model. 
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Figure  Bl:  Taxonomical  Fault  Frequency  in  Version  I  (light  grey)  versus 
random  distribution  of  faults  to  program  locations  (dark  grey). 
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Figure  B.2:  Taxonomical  Fault  Frequency  in  Version  11  (light  grey) 
versus  random  distribution  of  faults  to  program  locations  (dark  grey). 
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Figure  B.3:  Taxonomical  Fault  Frequency  in  Version  III  (light  grey) 
versus  random  distribution  of  faults  to  program  locations  (dark  grey). 
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Figure  B.4:  Taxonomical  Fault  Frequency  in  Version  IV  (light  grey) 
versus  random  distribution  of  faults  to  program  locations  (dark  grey). 


30 


26 


20 


16 


10 


LOOP 


OOKTROL 


PROCES. 


EXCEPTION 


Figure  B.5:  Taxonomical  Fault  Frequency  in  Version  V  (light  grey) 
versus  random  distribution  of  faults  to  program  locations  (dark  grey). 
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Figure  B.6:   Taxonomical  Fault  Frequency  in  Version  VI  (light  grey) 
versus  random  distribution  of  faults  to  program  locations  (dark  grey). 
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Figure  B.7:  Taxonomical  Fault  Frequency  in  Version  VU  (light  grey) 
versus  random  distribution  of  faults  to  program  locations  (dark  grey). 
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Figure  B.8:  Taxonomical  Fault  Frequency  in  Version  VIII  (light  grey) 
versus  random  distribution  of  faults  to  program  locations  (dark  grey). 
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Figure  B.9:  Actual  number  of  taxonomical  faults  to  expected  number  of 
faults  ratio  for  the  eight  versions  of  the  program. 
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APPENDIX  C:  SHARED  BOUNDING  DIMENSIONS  IN  TAXONOMI- 
CALLY  CLUSTERED  FAILURE  REGIONS 

In  this  Appendix  we  present  a  detailed  analysis,  whether  taxonomical 
clustering  of  failure  regions  results  in  an  increased  number  of  shared^ 
bounding  dimensions. 

The  results,  are  presented  in  the  tables  CI  to  C8.  Each  of  the  tables, 
corresponds  to  one  of  the  eight  versions  of  the  CONFLICT  program 
(Shimeall  1991,  1991b).  Each  table  entry,  corresponds  to  a  pair  of  failure 
regions  of  the  same  version.  It  is  noted  that  the  numbering  of  the  failure 
regions  is  not  significant  in  this  analysis.  It  merely  represents  the  order  in 
which  the  faults  and  the  corresponding  failure  regions  were  discovered. 

Table  entries  on  the  main  diagonal,  contain  the  logical  cluster  identifier 
(A  to  D),  for  the  corresponding  failure  region  (cf  Chapter  III).  Each  of  the 
off  diagonal   entries   contains   the   Jaccard   coefficient   for   the   identical 


'"^        In  this  analysis,  shared  dimensions  are  the  ones  that  correspond  to  the  same  predi- 
cates, identical. 
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dimensions  of  the  two  failure  regions  labeling  the  row  and  column,  and  the 
fault  type  identifier,  in  case  the  regions  belong  to  the  same  logical  cluster. 
For  example,  entry  (1.2,1.5)  contains  O.IOO./B  which  means  that  the 
Jaccard  Coefficient  for  these  failure  regions  is  equal  to  0.100.,  and  that  both 
correspond  to  faults  of  type  B.  On  the  other  hand,  entry  (1.12,1.5)  contains 
0.154,  so  the  two  failure  regions  have  a  Jaccard  coefficient  equal  to  0.154, 
but  correspond  to  different  types  of  faults  (D  and  B  respectively,  in  this 
case). 
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TABLE  C.l  : 
JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF  VERSION  I 


1.1 

1.2 

1.3 

1.4 

1.5 

1.6 

1.2 

0/B 

B 

- 

- 

- 

- 

1.3 

0 

0 

C 

- 

- 

- 

1.4 

0 

0 

1.333/C 

c 

- 

- 

1.5 

0/B 

0.100/B 

0 

0 

B 

- 

1.6 

0 

0 

0/C 

0/C 

0 

C 

1.7 

0 

0 

0/C 

0/C 

0 

0/C 

1.8 

0 

0.083/B 

0 

0 

0/B 

0 

1.9 

0/B 

0/B 

0 

0 

0/B 

0 

1.10 

0.333 

0 

0/C 

0/C 

0 

0/C 

1.11 

0.333 

0 

0/C 

0/C 

0 

0/C 

1.12 

0 

0.0833 

0 

0 

0.154 

0 

1.13 

0 

0 

0/C 

0/C 

0 

0/C 

1.14 

0/B 

0/B 

0 

0 

0/B 

0 

63 


TABLE  C.l  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  I 


1.7 

1.8 

1.9 

1.10 

1.11 

1.12 

1.13 

L7 

C 

- 

- 

- 

- 

- 

- 

1.8 

0 

B 

- 

- 

- 

- 

- 

1.9 

0 

0.063/B 

B 

- 

- 

- 

- 

1.10 

0/C 

0 

0 

c 

- 

- 

- 

1.11 

0/C 

0 

0 

0.580/C 

c 

- 

- 

1.12 

0 

0 

0 

0 

0 

D 

- 

1       ^'^^ 

0/C 

0.182 

0.083 

0/C 

0/C 

0 

C 

1.14 

0 

0.067/B 

0 

0 

0 

0 

0.143 

TABLE  C.l  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS 

OF  VERSION  I 


1.1 

1.2 

1.3 

1.4 

1.5 

1.6 

1.7 

1.17 

0.333\B 

0\B 

0 

0 

(>B 

0 

0 

1.18 

0 

0.100 

0 

0 

0.083 

0 

0 

1.19 

0 

0 

O^C 

OvC 

0.250 

CKC 

OvC 

1.20 

O^B 

0.05  6^B 

0 

0 

0.05  3\B 

0 

0 

1.22 

CKB 

0\B 

0 

0 

(hB 

0 

0 

1.23 

0 

0 

(XC 

(KC 

0 

(KC 

O^C 

1.25 

0 

0 

CNC 

(KC 

0.053 

(KC 

(KC 

1.27 

0 

0 

0\C 

(KC 

0 

(KC 

(KC 

1.28 

0 

0 

CKC 

(hC 

0.500 

CKC 

(KC 
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TABLE  C.l  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS 

OF  VERSION  I 


1.8 

1.9 

1.10 

1.11 

1.12 

1.13 

1.14 

1.17 

CNB 

0\B 

0.500 

0.500 

0 

0 

(KB 

1.18 

0 

0 

0 

0 

0.250\D 

0 

0.111 

1.19 

0 

0.154 

CNC 

CNC 

0.100 

0.125\C 

0.067 

1.20 

0.118\B 

0.04g\B 

0 

0 

0.053 

0.133 

0.105\B 

1.22 

0.056^ 

0.60CNB 

0 

0 

0 

0.071 

0.048\B 

1.23 

0 

0 

(KC 

OvC 

0.700 

(KC 

0 

1.25 

0.050 

0.045 

(hC 

CKC 

0 

o.imc 

0.109 

1.27 

0.120 

0.036 

(hC 

(KC 

0 

0.143\C 

0.074 

1.28 

0 

0 

0\C 

0\C 

0.143 

CNC 

0 
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TABLE  C.l  :    JACCARD  COEFFICIENTS  OF    FAILURE  REGIONS 

OF  VERSION  I 


1.17 

1.18 

1.19 

1.20 

L17 

B 

- 

- 

- 

1.18 

0 

D 

- 

- 

1.19 

0 

0.100 

C 

- 

1.20 

0/B 

0.100 

0.118 

B 

1.22 

0/B 

0 

0.067 

0.043\B 

1.23 

0 

0.250 

0.091\C 

0.050 

1.25 

0 

0 

0.059^C 

0.421 

1.27 

0 

0 

0.045\C 

0.103 

1             ^'^^ 

0 

0.071 

0.20CNC 

0.050 

TABLE  C.l  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS 

OF  VERSION  I 


1.22 

1.23 

1.25 

1.27 

1.28 

1.22 

B 

- 

- 

- 

- 

1.23 

0 

C 

- 

- 

- 

1.25 

0.042 

0.048\C 

C 

- 

- 

1.27 

0.033 

(KC 

0.06 1\C 

C 

- 

1.28 

0 

0.357\C 

0.05  3\C 

(KC 

C 
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TABLE  C.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.1 

2.2 

2.3 

2.4 

2.5 

2.6 

2.1 

B 

- 

- 

- 

- 

- 

2.2 

0/B 

B 

- 

- 

- 

- 

2.3 

0/B 

0.09 1/B 

B 

- 

- 

- 

2.4 

0 

0.091 

0.300 

C 

- 

- 

2.5 

0 

0.067 

0.133 

0.133/C 

C 

- 

2.6 

0 

0.091 

0.091 

0.083 

0.063 

D 

2.7 

0.077/B 

0/B 

0/B 

0 

0 

0 

2.8 

0.077/B 

0/B 

0/B 

0 

0 

0 

2.10 

0/B 

0/B 

0/B 

0 

0 

0 

2.11 

0/B 

0/B 

0/B 

0 

0 

0 

2.12 

0/B 

0/B 

0/B 

0 

0 

0 

2.13 

0/B 

0.111/B 

0.091/B 

0.091 

0.154 

0 

2.14 

0/B 

0/B 

0/B 

0 

0 

0 

2.15 

0/B 

0/B 

0/B 

0 

0 

0 
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TABLE  C.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.7 

2.8 

2.10 

2.11 

2.12 

2.13 

2.14 

2.15 

2.7 

B 

- 

- 

- 

- 

- 

- 

- 

2.8 

0.636/B 

B 

- 

- 

- 

- 

- 

- 

2.10 

0/B 

0/B 

B 

- 

- 

- 

- 

- 

2.11 

0/B 

0/B 

0/B 

B 

- 

- 

- 

- 

2.12 

0/B 

0/B 

0/B 

0/B 

B 

- 

- 

- 

2.13 

0/B 

0/B 

0/B 

0/B 

0/B 

B 

- 

- 

2.14 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

B 

- 

2.15 

0/B 

0/B 

0/B 

0/B 

0/B 

0.400/B 

0/B 

B 
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TABLE  C.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.1 

2.2 

2.3 

2.4 

2.5 

2.6 

2.16 

CNB 

(KB 

CKB 

0 

0 

0 

2.17 

(KB 

CKB 

(KB 

0 

0 

0 

2.18 

CKB 

(KB 

CKB 

0 

0 

0 

2.19 

CNB 

CKB 

CKB 

0 

0 

0 

2.20 

CNB 

(KB 

(KB 

0 

0 

0 

2.21 

(KB 

CKB 

CKB 

0 

0 

0 

2.22 

OvB 

(KB 

CKB 

0 

0 

0 

2.23 

CKB 

CKB 

CKB 

0 

0 

0 

2.25 

CNB 

CKB 

(KB 

0 

0 

0 

2.30 

CKB 

O.lllXB 

0.11 1\B 

0.100 

0.071 

0.333 
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TABLE  C.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.7 

2.8 

2.10 

2.11 

2.12 

2.13 

2.14 

2.15 

2.16 

0£ 

CNB 

OS 

CNB 

(KB 

CNB 

0.400/B 

0.40(KB 

2.17 

CNB 

O^B 

OvB 

(KB 

(KB 

CNB 

0.400/B 

0.400\B 

2.18 

OsB 

C^B 

(KB 

(KB 

(KB 

CNB 

0.400/B 

0.40CNB 

2.19 

OvB 

CNB 

OsB 

(KB 

(KB 

(KB 

CNB 

OVB 

2.20 

(hB 

CNB 

OsB 

(KB 

(KB 

CNB 

CNB 

(KB 

2.21 

CNB 

OB 

CNB 

CNB 

CNB 

CNB 

OVB 

(KB 

2.22 

0\B 

CNB 

(KB 

CNB 

CNB 

CNB 

CNB 

(XB 

2.23 

CNB 

CNB 

CNB 

(KB 

(KB 

CNB 

(KB 

OsB 

2.25 

CNB 

(XB 

CNB 

CNB 

(KB 

CNB 

(KB 

0\B 

2.30 

CNB 

CNB 

(KB 

(KB 

CNB 

CNB 

(KB 

CNB 
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TABLE  C.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.16 

2.17 

2.18 

2.19 

2.20 

2.16 

B 

- 

- 

- 

- 

2.17 

0.400/B 

B 

- 

- 

- 

2.18 

0/B 

0.400/B 

B 

- 

- 

2.19 

0/B 

0/B 

0/B 

B 

- 

2.20 

0/B 

0/B 

0/B 

0.571/B 

B 

2.21 

0/B 

0/B 

0/B 

0.571/B 

0.571/B 

2.22 

0/B 

0/B 

0/B 

0.571/B 

0.571/B 

2.23 

0/B 

0/B 

0/B 

0.571/B 

0.571/B 

2.25 

0/B 

0/B 

0/B 

0/B 

0/B 

2.30 

0/B 

0/B 

0/B 

0/B 

0/B 

TABLE  C.2  :   JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.21 

2.22 

2.23 

2.24 

2.30 

2.21 

B 

- 

- 

- 

- 

2.22 

0.571/B 

B 

- 

- 

- 

2.23 

0.571/B 

0.571/B 

B 

- 

_ 

2.25 

0/B 

0/B 

0/B 

B 

- 

2.30 

0/B 

0/B 

0/B 

0/B 

B 
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TABLE  C.3  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS 

OF  VERSION  m 


3.1 

3.2 

3.3 

3.4 

3.5 

3.6 

3.7 

3.9 

3.10 

3.1 

B 

- 

- 

- 

- 

- 

- 

- 

- 

3.2 

0.600- 
/B 

B 

- 

- 

- 

- 

- 

- 

- 

3.3 

0.250- 
/B 

0.286- 
/B 

B 

- 

- 

- 

- 

- 

- 

3.4 

0/B 

0/B 

0/B 

B 

- 

- 

- 

- 

- 

3.5 

0/B 

0/B 

0/B 

0/B 

B 

- 

- 

- 

- 

3.6 

0.143 

0.200 

0 

0 

0 

A 

- 

- 

- 

3.7 

0.067 

0.071 

0 

0 

0 

0.071 

- 

- 

- 

3.9 

0 

0 

0 

0 

0 

0 

0.476 

C 

- 

3.10 

0 

0 

0 

0 

0.167 

0 

0 

0/C 

C 

3.11 

O.lll/B 

0.125/B 

0.091/B 

0/B 

0/B 

0.125 

0.059 

0 

0 

3.12 

O.lll/B 

0.125/B 

0/B 

0/B 

0/B 

0.11 

0.188 

0.087 

0 

3.13 

0 

0 

0 

0 

0 

0 

0 

0.042/C 

0/C 

3.15 

0.077 

0.083 

0.364 

0.100 

0 

0 

0 

0.040/C 

0/C 

3.16 

0.11 

0.125 

0.091 

0 

0 

0.125 

0.059 

0 

0 

3.17 

0\B 

0/B 

0/B 

0/B 

0/B 

0 

0 

0 

0 

3.18 

0.067 

0.077 

0 

0 

0 

0.091 

0.714 

0.429/C 

0/C 

3.21 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3.22 

0.125- 
/B 

0.167- 
/B 

0/B 

0/B 

0/B 

0.200 

0.063 

0.048 

0 

3.23 

0.182/B 

0.200/B 

0.077/B 

0/B 

0.0833/B 

0.091 

0.111 

0.040 

0.111 
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TABLE  C.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  in 


3.11 

3.12 

3.13 

3.15 

3.16 

3.17 

3.18 

3.21 

3.22 

3.2 

3 

3.11 

B 

- 

- 

- 

- 

- 

- 

- 

- 

- 

3.12 

0.111- 
/B 

B 

- 

- 

- 

- 

- 

- 

- 

- 

3.13 

0 

0 

C 

- 

- 

- 

- 

- 

- 

- 

3.15 

0.067 

0 

0/C 

c 

- 

- 

- 

- 

- 

- 

3.16 

0.333 

0.111 

0 

0.067 

D 

- 

- 

- 

- 

- 

3.17 

0/B 

0/B 

0 

0 

0 

B 

- 

- 

- 

- 

3.18 

0.063 

0.059 

0/C 

0.056 

0.063 

0 

C 

- 

- 

- 

3.21 

0 

0 

0 

0 

0/D 

0 

0 

D 

- 

- 

3.22 

0.111- 
/B 

0.100- 
/B 

0 

0 

0.111 

0/B 

0.154 

0 

B 

- 

3.23 

0.077- 
/B 

0.167- 
/B 

0 

0.063 

0 

0 

0.053 

0 

0.833- 
/B 

B 
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TABLE  C.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  ra 


3.1 

3.2 

3.3 

3.4 

3.5 

3.6 

3.7 

3.9 

3.10 

3.24 

0.143 

0.167 

0.167 

0 

0.286 

0 

0.071 

0/C 

0.250- 

/c 

3.25 

0.125 

0.167 

0.250 

0 

0 

0.250 

0.067 

0/C 

0/C 

3.26 

0/B 

0/B 

0.125/B 

0.125/B 

0.571/B 

0 

0 

0 

0.167 

3.28 

0/B 

0.125/B 

0/B 

0.100/B 

0.222/B 

0.125 

0.063 

0 

0 

3.32 

0/B 

0/B 

0.167/B 

0/B 

0/B 

0 

0 

0 

0 

3.33 

0/B 

0/B 

0.143/B 

0/B 

0/B 

0 

0 

0 

0 

3.34 

0/B 

0/B 

0.143/B 

0/B 

0/B 

0 

0 

0 

0 

3.35 

0/B 

0/B 

0.143/B 

0/B 

0/B 

0 

0 

0 

0 

3.36 

0/B 

0/B 

0.167/B 

0/B 

0/B 

0 

0 

0 

0 

3.37 

0/B 

0/B 

0.143/B 

0/B 

0/B 

0 

0 

0 

0 

3.38 

O.lll/B 

0/B 

0/B 

0.125/B 

0.222/B 

0 

0.067 

0 

0.167 

3.39 

0/B 

0/B 

0/B 

0.125/B 

0.222/B 

0 

0 

0 

0.200 

3.40 

0/B 

0/B 

0/B 

0.125/B 

0.222/B 

0 

0 

0 

0.200 

3.41 

0.071/B 

0/B 

0/B 

0.125/B 

0.222/B 

0 

0 

0 

0.200 

3.42 

0/B 

O.lll/B 

0/B 

0.126/B 

0.222/B 

0 

0 

0 

0.200 

3.43 

0.091/B 

0/B 

0/B 

0.167/B 

0/B 

0.143 

0.118 

0.042 

0 

3.44 

0/B 

0.083/B 

0/B 

0/B 

0/B 

0.091 

0.770 

0.381 

0 

3.45 

0/B 

0/B 

0/B 

0/B 

0.056/B 

0 

0 

0.529 

0.381 
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TABLE  C.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  ffl 


3.11 

3.12 

3.13 

3.15 

3.16 

3.17 

3.18 

3.21 

3.22 

3.23 

3.24 

0 

0 

0/C 

0.10- 
0/C 

0 

0 

0.07- 
7/C 

0 

0 

0.18 

2 

3.25 

0.111 

0.100 

0/C 

0.09- 
1/C 

0.11 
1 

0 

0.08- 
3/C 

0 

0.16 

7 

0.08 

3 

3.26 

0/B 

0/B 

0 

0.091 

0 

0/B 

0 

0 

0/B 

0.0- 
83- 
/B 

3.28 

0.11- 
1/B 

0.11- 
1/B 

0 

0 

0.11 

1 

0/B 

0.063 

0 

0.11- 
1/B 

0.0- 
83- 
/B 

3.32 

0/B 

0/B 

0 

0.100 

0 

0/B 

0 

0.33 
3 

0/B 

0/B 

3.33 

0/B 

0/B 

0 

0.091 

0 

0/B 

0 

0.25 
0 

0/B 

0/B 

3.34 

0/B 

0/B 

0 

0.091 

0 

0/B 

0 

0.25 
0 

0/B 

0/B 

3.35 

0/B 

0/B 

0 

0.091 

0 

0/B 

0 

0.25 
0 

0/B 

0/B 
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TABLE  C.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  in 


3.11 

3.12 

3.13 

3.15 

3.16 

3.17 

3.18 

3.21 

3.22 

3.23 

3.36 

0/B 

0/B 

0 

0.09 

1 

0 

0/B 

0 

0.25 
0 

0/B 

0/B 

3.37 

0/B 

0/B 

0 

0.09 

1 

0 

0/B 

0 

0.25 
0 

0/B 

0/B 

3.38 

0/B 

0/B 

0 

0 

0 

0/B 

0.063 

0 

0/B 

0.08- 
3/B 

3.39 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0/B 

0.08- 
3/B 

3.40 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0/B 

0.08- 
3/B 

3.41 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0/B 

0.08- 
3/B 

3.42 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0/B 

0.08- 

3/B 

3.43 

0.08- 
3/B 

0.07- 
7/B 

0 

0 

0.08 
3 

0/B 

0.143 

0 

0.111- 
/B 

0.06- 
7/B 

3.44 

0.06- 

7/B 

0.06- 
3/B 

0 

0 

0.06 

7 

0/B 

0.910 

0 

0.077- 
/B 

0.05- 
6/B 

3.45 

0/B 

0.05- 
0/B 

0 

0 

0 

0/B 

0.471 

0 

0/B 

0/B 
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TABLE  C.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  ffl 


3.24 

3.25 

3.26 

3.28 

3.32 

3.33 

3.34 

3.35 

3.24 

C 

- 

- 

- 

- 

- 

- 

3.25 

0 

C 

- 

- 

- 

- 

- 

- 

3.26 

0.333 

0 

B 

- 

- 

- 

- 

_ 

3.28 

0.125 

0.111 

0.333- 
/B 

B 

- 

- 

- 

- 

3.32 

0 

0 

0.167/B 

0/B 

B 

- 

- 

- 

3.33 

0 

0 

0.143/B 

0/B 

0.500/B 

B 

- 

- 

3.34 

0 

0 

0.143/B 

0/B 

0.500/B 

0.400/B 

B 

- 

3.35 

0 

0 

0.143/B 

0/B 

0.500/B 

0.400/B 

0.400/B 

B 

3.36 

0 

0 

0.143/B 

0/B 

0.500/B 

0.400/B 

0.400/B 

0.400/B 

3.37 

0 

0 

0.143/B 

0/B 

0.500/B 

0.400/B 

0.400/B 

0.400/B 

3.38 

0.286 

0 

0.375/B 

0.200/B 

0/B 

0/B 

0/B 

0/B 

3.39 

0.143 

0 

0.250/B 

0.100/B 

0/B 

0/B 

0/B 

0/B 

3.40 

0.143 

0 

0.250/B 

0.100/B 

0/B 

0/B 

0/B 

0/B 

3.41 

0.143 

0 

0.250/B 

0.100/B 

0/B 

0/B 

0/B 

0/B 

3.42 

0.143 

0 

0.250/B 

0.100/B 

0/B 

0/B 

0/B 

0/B 

3.43 

0 

0.125 

0/B 

0.083/B 

0/B 

0/B 

0/B 

0/B 

3.44 

0.077 

0.083 

0/B 

0.063/B 

0/B 

0/B 

0/B 

0/B 

3.45 

0.059 

0 

0.059/B 

0.053/B 

0/B 

0/B 

0/B 

0/B 
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TABLE  C.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  in 


3.36 

3.37 

3.38 

3.39 

3.40 

3.41 

3.42 

3.43 

3.44 

3.45 

3.36 

B 

- 

- 

- 

- 

- 

- 

- 

- 

- 

3.37 

0.400- 
/B 

B 

- 

- 

- 

- 

- 

- 

- 

- 

3.38 

0/B 

0/B 

B 

- 

- 

- 

- 

- 

- 

- 

3.39 

0/B 

0/B 

0.286/B 

B 

- 

- 

- 

- 

- 

- 

3.40 

0/B 

0/B 

0.286/B 

0.500/B 

B 

- 

- 

- 

- 

- 

3.41 

0/B 

0/B 

0.286/B 

0.500/B 

0.500- 
/B 

B 

- 

- 

- 

- 

3.42 

0/B 

0/B 

0.286/B 

0.500/B 

0.500- 
/B 

0.500/ 
B 

B 

- 

- 

- 

3.43 

0/B 

0/B 

0/B 

0.091/B 

0.091/ 
B 

0.091/ 
B 

0.091/ 
B 

B 

- 

- 

3.44 

0/B 

0/B 

0.067/B 

0/B 

0/B 

0/B 

0/B 

0.143 
/B 

B 

- 

3.45 

0/B 

0/B 

0.105/B 

0.053/B 

0.053/ 
B 

0.053/ 
B 

0.053/ 
B 

0.053 
/B 

0/B 

B 
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TABLE  C.4  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS 


OF  VERSION  IV 


4.1 

4.2 

4.3 

4.4 

4.5 

4.7 

4.8 

4.9 

4.10 

4.11 

4.12 

4.1 

C 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.2 

0 

B 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.3 

0/C 

0.118 

C 

- 

- 

- 

- 

- 

- 

- 

- 

4.4 

0/C 

0 

0.045/ 
C 

c 

- 

- 

- 

- 

- 

- 

- 

4.5 

0 

0/B 

0.063 

0.071 

B 

- 

- 

- 

- 

- 

- 

4.7 

0 

0/B 

0.059 

0.071 

0.167/ 
B 

B 

- 

- 

- 

- 

- 

4.8 

0 

0/B 

0.167 

0.059 

0.125/ 
B 

0.111/ 
B 

B 

- 

- 

- 

- 

4.9 

0/C 

0 

0.063/ 
C 

0.071/ 
C 

0.250 

0.167 

0.125 

C 

- 

- 

- 

4.10 

0 

0/B 

0.056 

0.063 

0/B 

0.125/ 
B 

0.091/ 
B 

0.125 

B 

- 

- 

4.11 

0/C 

0.053 

0.043/ 
C 

0/C 

0 

0 

0 

0/C 

0 

C 

- 

4.12 

0/C 

0 

0.063/ 
C 

0.214/ 
C 

0.167 

0.125 

0.100 

0.167/ 
C 

0.100 

0/C 

c 

4.13 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4.14 

0/C 

0 

0/C 

0/C 

0 

0 

0 

0/C 

0 

0/C 

0/C 

4.15 

0.500 

0/B 

0.091 

0 

0/B 

0/B 

0.214/ 
B 

0 

0/B 

0 

0 

4.16 

0 

0.071/ 
B 

0 

0.188 

0/B 

0/B 

0/B 

0 

0/B 

0 

0.077 

4.18 

0/C 

0.100 

0/C 

0/C 

0 

0 

0 

0/C 

0 

0/C 

0/C 

4.19 

0 

0.100 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4.21 

0 

0.071 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4.22 

0 

0/B 

0 

0 

0/B 

0/B 

0/B 

0 

0/B 

0 

0 

4.24 

0/C 

0.100 

0/C 

0 

0 

0 

0 

0/C 

0 

0/C 

0/C 

4.26 

0/C 

0.083 

0/C 

0 

0 

0 

0 

0/C 

0 

0/C 

0/C 
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TABLE  C.4  :    JACCARD  COEFFICIENTS  OF    FAILURE  REGIONS 

OF  VERSION  IV 


4.13 

4.14 

4.15 

4.16 

4.18 

4.19 

4.21 

4.22 

4.24 

4.2 
8 

4.13 

A 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.14 

0.800 

c 

- 

- 

- 

- 

- 

- 

- 

- 

4.15 

0 

0 

B 

- 

- 

- 

- 

- 

- 

- 

4.16 

0 

0 

0/B 

B 

- 

- 

- 

- 

- 

- 

4.18 

0 

0/C 

0 

0.091 

C 

- 

- 

- 

- 

- 

4.19 

0 

0 

0 

0.071 

0.125 

D 

- 

- 

- 

- 

4.21 

0 

0 

0 

0.063 

0.083 

0.077/ 
D 

D 

- 

- 

- 

4.22 

0 

0 

0/B 

0 

0 

0 

0 

B 

- 

- 

4.24 

0 

0/C 

0 

0.083 

0.167/C 

0.143 

0 

0 

c 

- 

4.26 

0 

0/C 

0 

0.071 

0.125/C 

0.111 

0.077 

0 

0.60 

/c 

C 
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TABLE  C5:    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 


VERSION  V 


5.1 

5.2 

5.3 

5.4 

5.5 

5.6 

5.7 

5.8 

5.9 

5.1 

B 

- 

- 

- 

- 

- 

- 

- 

5.2 

(hB 

B 

- 

- 

- 

- 

- 

- 

- 

5.3 

0\B 

(KB 

B 

- 

- 

- 

- 

- 

- 

5.4 

CNB 

0.053\B 

(XB 

B 

- 

- 

- 

- 

- 

5.5 

0 

0.059 

0 

0.375 

C 

- 

- 

- 

- 

5.6 

0\B 

OsB 

0.167\B 

O^B 

0 

B 

- 

- 

- 

5.7 

CKB 

(KB 

CNB 

0\B 

0 

0\B 

B 

- 

- 

5.8 

0 

0.111 

0 

0.059 

0.06AC 

0 

0 

C 

- 

5.9 

(KB 

0.053\B 

0\B 

(XB 

0 

0^ 

O^B 

0 

B 

5.10 

(KB 

(KB 

0\B 

CNB 

0 

(KB 

(KB 

0 

0.143\B 

5.11 

(KB 

(KB 

O^B 

(KB 

0 

OsB 

0\B 

0 

OB 

5.12 

(KB 

0\B 

0\B 

CNB 

0 

OvB 

CNB 

0 

0\B 

5.13 

0 

0.050 

0 

0.083 

0.100 

0 

0 

0.056 

0 

5.15 

(KB 

0\B 

(KB 

0\B 

0 

(KB 

OS 

0 

0\B 

5.16 

(KB 

CNB 

(KB 

(KB 

0 

(KB 

0\B 

0 

CNB 

5.17 

(KB 

0\B 

CNB 

0^ 

0 

CNB 

0\B 

0 

OvB 

5.18 

(KB 

OvB 

O^B 

O^B 

0 

CNB 

O^B 

0 

(KB     1 

5.19 

(KB 

OS 

CNB 

(KB 

0 

CNB 

0\B 

0 

OS 
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TABLE  C5:  JACCARD  COEFFICIENTS  OF    FAILURE  REGIONS  OF 

VERSION  V 


5.10 

5.11 

5.12 

5.13 

5.15 

5.16 

5.1 

7 

5.18 

5.17 

5.10 

B 

- 

- 

- 

- 

- 

- 

- 

- 

5.11 

OvB 

B 

- 

- 

- 

- 

- 

- 

- 

5.12 

(KB 

0.200V 
B 

B 

- 

- 

- 

- 

- 

- 

5.13 

0 

0 

0 

D 

- 

- 

- 

- 

- 

5.15 

OvB 

(KB 

OvB 

0 

B 

- 

- 

- 

- 

5.16 

OVB 

CKB 

OvB 

0 

0.714 
\B 

B 

- 

- 

- 

5.17 

(XB 

OvB 

(KB 

0 

OvB 

(KB 

B 

- 

- 

5.18 

OvB 

OvB 

OvB 

0 

OvB 

(KB 

OvB 

B 

- 

5.19 

OvB 

OvB 

OvB 

0 

OvB 

OvB 

OvB 

OvB 

B 
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TABLE  C5:  JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  V 


5.1 

5.2 

5.3 

5.4 

5.5 

5.6 

5.7 

5.8 

5.20 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.21 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.22 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.23 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.24 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.25 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.26 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.27 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.28 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.30 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.31 

0 

0 

0 

0 

0/C 

0 

0 

0 

5.32 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.33 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.34 

0 

0 

0 

0 

0/C 

0 

0 

0 

5.35 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

5.37 

0 

0 

0 

0 

0 

0 

0 

0 

5.39 

0 

0 

0 

0 

0 

0 

0 

0 

5.40 

0 

0 

0 

0 

0 

0 

0 

0      1 
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TABLE  C5:  JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  V 


5.9 

5.10 

5.11 

5.12 

5.13 

5.15 

5.16 

5.1 

7 

5.18 

5.19 

5.20 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.21 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.22 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.23 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.24 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.25 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.26 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.27 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.28 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.30 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.31 

0 

0 

0 

0 

0 

0.130 

0.261 

0 

0 

0 

5.32 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.33 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

5.34 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.35 

0/B 

0/B 

0/B 

0/B 

0 

0.040/ 
B 

0.040/ 
B 

0/B 

0/B 

0/B 

5.37 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.39 

0 

0 

0 

0 

0 

0.083 

0.083 

0 

0 

0 

5.40 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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TABLE  C5:   JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  V 


5.19 

5.20 

5.21 

5.22 

5.23 

5.24 

5.25 

5.26 

5.27 

5.20 

0/B 

B 

- 

- 

- 

- 

- 

- 

- 

5.21 

0.444/ 
B 

0/B 

B 

- 

- 

- 

- 

- 

- 

5.22 

0/B 

0.286/B 

0/B 

B 

- 

- 

- 

- 

- 

5.23 

0.444/ 
B 

0/B 

0.444/ 
B 

0/B 

B 

- 

- 

- 

- 

5.24 

0/B 

0.286/B 

0/B 

0.286/ 
B 

0/B 

B 

- 

- 

- 

5.25 

0.444/ 
B 

0/B 

0.444/ 
B 

0/B 

0.444/ 
B 

0/B 

B 

- 

- 

5.26 

0/B 

0.286/B 

0/B 

0.286/ 
B 

0/B 

0.286/ 
B 

0/B 

B 

- 

5.27 

0.444/ 
B 

0/B 

0.444/ 
B 

0/B 

0.444/ 
B 

0/B 

0.444 
/B 

0/B 

B 

5.28 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

5.30 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

5.31 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.32 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

5.33 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

5.34 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.35 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

5.37 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.39 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.40 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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TABLE  C5:    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  V 


5.28 

5.30 

5.31 

5.32 

5.33 

5.34 

5.35 

5.37 

5.39 

5.28 

B 

- 

- 

- 

- 

- 

- 

- 

- 

5.30 

0/B 

B 

- 

- 

- 

- 

- 

- 

- 

5.31 

0 

0 

C 

- 

- 

- 

- 

- 

- 

5.32 

0/B 

0/B 

0/B 

B 

- 

- 

- 

- 

- 

5.33 

0/B 

0/B 

0/B 

0/B 

B 

- 

- 

- 

- 

5.34 

0 

0 

0/C 

0 

0 

C 

- 

- 

- 

5.35 

0/B 

0/B 

0.053/ 
B 

0/B 

0/B 

0.214/ 
B 

B 

- 

- 

5.37 

0 

0 

0 

0 

0 

0 

0 

- 

- 

5.39 

0 

0 

0.125 

0 

0 

0 

0.063 

0 

- 

5.40 

0 

0 

0 

0 

0 

0 

0 

0 

0.053 
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TABLE  C.6  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 


VERSION  VI 


6.1 

6.2 

6.3 

6.4 

6.5 

6.6 

6.7 

6.8 

6.1 

C 

- 

- 

- 

- 

- 

- 

- 

6.2 

0 

B 

- 

- 

- 

- 

- 

- 

6.3 

0.357/ 
C 

0 

C 

- 

- 

- 

- 

- 

6.4 

0 

0 

0 

A 

- 

- 

- 

- 

6.5 

0/C 

0 

0/C 

0 

C 

- 

- 

- 

6.6 

0.077 

0 

0.063 

0 

0 

D 

- 

- 

6.7 

0.056/ 
C 

0 

0.048/ 
C 

0 

0/C 

0.091 

C 

- 

6.8 

0.167/ 
C 

0.143 

0/C 

0 

0/C 

0.091 

0.500/ 
C 

C 

6.9 

0.059/C 

0 

0.050/C 

0 

0/C 

0.100 

0.286/C 

0.286/C 

6.11 

0/C 

0.143 

0/C 

0 

0/C 

0 

0/C 

0/C 

6.12 

0/C 

0 

0/C 

0 

0/C 

0 

0/C 

0/C 

6.13 

0 

0/B 

0 

0 

0 

0 

0 

0 

1     ^'^^ 

0/C 

0 

0/C 

0 

0/C 

0 

0/C 

0/C 

6.15 

0/C 

0.111 

0/C 

0 

0/C 

0 

0/C 

0/C 

6.16 

0.364/C 

0 

0.286/C 

0 

0/C 

0.125 

0.083/C 

0.250/C 

6.17 

0.056 

/B 

0.048 

0 

0 

0.083 

0.091 

0.091 

1     6.18 

0/C 

0 

0/C 

0 

0/C 

0 

0/C 

0/C 
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TABLE  C.6  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  VI 


6.1 


6.2 


6.3 


6.4 


6.5 


6.6 


6.7 


6.8 


6.20 


0 


0/B 


6.23 


0.056 


0/B 


0.048 


0.091 


0.091 


0.091 


6.24 


0/C 


0 


0/C 


0 


0/C 


0 


0/C 


0/C 


TABLE  C.6 


JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 
VERSION  VI 


6.9 

6.11 

6.12 

6.13 

6.14 

6.1 
5 

6.1 
6 

6.1 

7 

6.18 

6.20 

6.2 
3 

6.2 

4 

6.9 

C 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

6.11 

0/C 

C 

- 

- 

- 

- 

- 

- 

- 

- 

- 

" 

6.12 

0/C 

0/C 

C 

- 

- 

- 

- 

- 

- 

- 

- 

- 

6.13 

0 

0 

0 

B 

- 

- 

- 

- 

- 

- 

- 

- 

6.14 

0/C 

0/C 

0.059/ 

c 

0 

C 

- 

- 

- 

- 

- 

- 

- 

6.15 

0/C 

0.143/ 

c 

0.071/ 
C 

0 

0/C 

C 

- 

- 

- 

- 

- 

- 

6.16 

0.07 
1/C 

0/C 

0/C 

0 

0/C 

0/C 

C 

- 

- 

- 

- 

. 

6.17 

0.06 

7 

0 

0 

0/B 

0 

0 

0.0 
77 

B 

- 

- 

- 

- 

6.18 

0/C 

0/C 

0.06 
7/C 

0 

0.40 
0/C 

0/C 

0/C 

0 

C 

- 

- 

- 

6.20 

0 

0 

0 

0/B 

0 

0 

0 

0/B 

0 

B 

- 

- 

6.23 

0.07 
1 

0 

0 

0/B 

0 

0 

0.0 

77 

0.73/ 
B 

0 

0/ 
B 

B 

- 

6.24 

0/C 

0.071/ 

c 

0.118/ 
C 

0 

0.27 
8/C 

0.063 

/c 

0/C 

0 

0.46 

7/C 

0 

0 

C 
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TABLE  C.7:    JACCARD  COEFFICIENTS  OF    FAILURE  REGIONS  OF 


VERSION  vn 


7.1 

7.2 

7.3 

7.4 

7.5 

7.6 

7.7 

7.1 

B 

- 

- 

- 

- 

- 

- 

7.2 

0.083/B 

B 

- 

- 

- 

- 

- 

7.3 

0.083/B 

0.818/B 

B 

- 

- 

- 

. 

7.4 

0/B 

0/B 

0/B 

B 

- 

- 

- 

II       7.5 

0.167 

0 

0 

0.071 

C 

- 

- 

7.6 

0/B 

0/B 

0/B 

0/B 

0 

B 

- 

7.7 

0/B 

0/B 

0/B 

0/B 

0 

0.333/B 

B 

7.8 

0/B 

0/B 

0/B 

0/B 

0 

0.333/B 

0.500/B 

7.9 

0 

0 

0 

0.333 

0 

0 

0 

7.10 

0 

0.231 

0.214 

0 

0/C 

0 

0 

7.11 

0 

0 

0 

0 

0 

0.200 

0.167 

7.12 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

7.13 

0 

0 

0 

0 

0 

0 

0 

7.14 

0 

0 

0 

0.143 

0/C 

0 

0 

7.15 

0.200 

0.091 

0.083 

0 

0/C 

0 

0 

7.16 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

0/B 
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TABLE  C.7:    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.8 

7.9 

7.10 

7.11 

7.12 

7.13 

7.14 

7.15 

7.16 

7.8 

B 

- 

- 

- 

- 

- 

- 

- 

- 

7.9 

0 

- 

- 

- 

- 

- 

- 

- 

- 

7.10 

0 

0 

c 

- 

- 

- 

- 

- 

- 

7.11 

0.167 

0 

0 

D 

- 

- 

- 

- 

- 

7.12 

0/B 

0 

0 

0 

B 

- 

- 

- 

- 

7.13 

0 

0 

0 

0 

0 

A 

- 

- 

- 

7.14 

0 

0 

0/C 

0 

0.100 

0 

C 

- 

- 

7.15 

0 

0 

0/C 

0 

0 

0 

0/C 

C 

7.16 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

B 
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TABLE  C.7;   JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.1 

7.2 

7.3 

7.4 

7.5 

7.6 

7.17 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.18 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.19 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.20 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.21 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.22 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.23 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.24 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.25 

0/B 

0/B 

0/B 

0/B 

0 

0/B 

7.27 

0.100 

0.200 

0.273 

0 

0 

0 

7.28 

0 

0 

0 

0 

0/C 

0 

7.29 

0.077 

0.455 

0.417 

0 

0/C 

0 

7.32 

0.048 

0.038 

0.037 

0 

0/C 

0 

7.33 

0 

0 

0 

0 

0 

0.143 

7.35 

0.077 

0.333 

0.400 

0 

0/C 

0 
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TABLE  C.7:  JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.7 

7.8 

7.9 

7.10 

7.11 

7.12 

7.13 

7.1 
4 

7.15 

7.16 

7.17 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0.400/B 

7.18 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0.400/B 

7.19 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0.400/B 

7.20 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0.400/B 

7.21 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0/B 

7.22 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0/B 

7.23 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0/B 

7.24 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0/B 

7.25 

0/B 

0/B 

0 

0 

0 

0/B 

0 

0 

0 

0/B 

7.27 

0 

0 

0 

0.07 
7 

0/D 

0 

0 

0 

0.111 

0 

7.28 

0 

0 

0 

0/C 

0 

0 

0 

0/C 

0/C 

0 

7.29 

0 

0 

0 

0.231/ 

c 

0 

0 

0 

0/C 

0.083/C 

0 

7.32 

0 

0 

0 

0/C 

0 

0 

0 

0/C 

0.050/C 

0 

7.33 

0.125 

0.125 

0 

0 

0.333/ 
D 

0 

0 

0 

0 

0 

7.35 

0 

0 

0 

0.200/ 

c 

0 

0 

0 

0/C 

0.091/ 

c 

0 
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TABLE  C.7:  JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.17 

7.18 

7.19 

7.20 

7.21 

1          7.17 

B 

- 

- 

- 

- 

1          ^-1* 

0.400/B 

B 

- 

- 

- 

7.19 

0.400/B 

0.400/B 

B 

- 

- 

7.20 

0.400/B 

0.400/B 

0.400/B 

B 

- 

7.21 

0/B 

0/B 

0/B 

0/B 

B 

7.22 

0/B 

0/B 

0/B 

0/B 

0.571/B 

7.23 

0/B 

0/B 

0/B 

0/B 

0.57 1/B 

7.24 

0/B 

0/B 

0/B 

0/B 

0.571/B 

7.25 

0/B 

0/B 

0/B 

0/B 

0.429/B 

7.27 

0 

0 

0 

0 

0 

7.28 

0 

0 

0 

0 

0 

7.29 

0 

0 

0 

0 

0 

7.32 

0 

0 

0 

0 

0 

7.33 

0 

0 

0 

0 

0 

7.35 

0 

0 

0 

0 

0 
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TABLE  C.7:  JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.22 

7.23 

7.24 

7.25 

7.27 

7.28 

7.29 

7.32 

7.3 
3 

7.35 

7.22 

B 

- 

- 

- 

- 

- 

- 

- 

- 

- 

7.23 

0.571/ 
B 

B 

- 

- 

- 

- 

- 

- 

- 

- 

7.24 

0.571/ 
B 

0.571/ 
B 

B 

- 

- 

- 

- 

- 

- 

- 

7.25 

0.429/ 
B 

0.429/ 
B 

0.429/ 
B 

B 

- 

- 

- 

- 

- 

- 

7.27 

0 

0 

0 

0 

D 

- 

- 

- 

- 

- 

7.28 

0 

0 

0 

0 

0 

C 

- 

- 

- 

- 

7.29 

0 

0 

0 

0 

0.182 

0/C 

c 

- 

- 

- 

7.32 

0 

0 

0 

0 

0.125 

0/C 

0.037/ 

c 

C 

- 

- 

7.33 

0 

0 

0 

0 

0/D 

0 

0 

0 

D 

- 

7.35 

0 

0 

0 

0 

0.200 

0/C 

0.333/ 
C 

0.034/ 
C 

0 

C 
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TABLE  C.8:    JACCARD  COEFHCIENTS  OF    FAILURE  REGIONS  OF 


VERSION  vin 


8.1 

8.2 

8.3 

8.4 

8.5 

8.6 

8.7 

8.8 

8.1 

D 

- 

- 

- 

- 

- 

- 

- 

8.2 

0/D 

D 

- 

- 

- 

- 

- 

- 

8.3 

0/D 

0.500/ 
D 

D 

- 

- 

- 

- 

- 

8.4 

0/D 

0.143/ 
D 

0.250/ 
D 

D 

- 

- 

- 

. 

8.5 

0 

1.00 

0.500 

0.429 

- 

- 

- 

- 

8.6 

0 

0.500 

0 

0.125 

0.500 

B 

- 

- 

8.7 

0 

0.333 

0.667 

0.111 

0.333 

0.667 

C 

- 

8.8 

0.250/D 

0/D 

0/D 

0/D 

0 

0 

0 

D 

8.9 

0.125 

0 

0 

0 

0 

0 

0 

0 

8.11 

0 

0.333 

0.200 

0.100 

0.250 

0.200/B 

0.167 

0.667 

8.12 

0 

0 

0 

0 

0 

0/B 

0 

0 

8.13 

0.750 

0 

0 

0 

0 

0/B 

0 

0.500 

8.14 

0.222/D 

0/D 

0/D 

0/D 

0 

0 

0 

0/D 

8.15 

0.750 

0 

0 

0 

0 

0/B 

0 

1.00 

8.18 

0.143 

0 

0 

0 

0 

0/B 

0 

0 

8.19 

0.286 

0 

0 

0 

0 

0/B 

0 

0.500 

8.23 

0 

0.500 

0.333 

0.111 

0.333 

0.333/B 

0.250 

1.00 

8.25 

0 

0 

0 

0 

0 

0/B 

0 

0 

8.26 

0 

0 

0 

0 

0 

0/B 

0 

0 
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TABLE  C.8:  JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vra 


8.9 

8.11 

8.12 

8.13 

8.14 

8.15 

8.18 

8.19 

8.2 

3 

8.25 

8.2 
6 

8.9 

A 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.11 

0 

B 

- 

- 

- 

- 

- 

- 

- 

- 

8.12 

0 

0/B 

B 

- 

- 

- 

- 

- 

- 

- 

- 

8.13 

0.167 

0/B 

0/B 

B 

- 

- 

- 

- 

- 

- 

- 

8.14 

0.091 

0 

0 

0.12 
5 

D 

- 

- 

- 

- 

- 

- 

8.15 

0.200 

0/B 

0/B 

1.00 

0.429 

B 

- 

- 

- 

- 

- 

8.18 

0 

0/B 

0/B 

0 

0.100 

0.25(N 
B 

B 

- 

- 

- 

- 

8.19 

0.125 

0.167/B 

0/B 

0.40 
0 

0.200 

0.75(A 
B 

0\B 

B 

- 

- 

- 

8.23 

0 

0.500/B 

0/B 

0 

0 

0\B 

0\B 

0\B 

B 

- 

- 

8.25 

0 

0/B 

0/B 

0 

0 

(hB 

OsB 

CNB 

0\B 

B 

- 

8.26 

0 

0/B 

0/B 

0 

0 

0\B 

(m 

(KB 

CNB 

0.66 
7VB 

B 
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TABLE  C.8:  JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vra 


8.1 

8.2 

8.3 

8.4 

8.5 

8.6 

8.7 

8.8 

8.9 

8.27 

0 

0 

0 

0 

0 

0/B 

0 

0 

0 

8.28 

0 

0 

0 

0 

0 

0/B 

0 

0 

0 

8.29 

0 

0 

0 

0 

0 

0/B 

0 

0 

0 

8.30 

0 

0 

0 

0 

0 

0/B 

0 

0 

0 

8.31 

0 

0 

0 

0 

0 

0/B 

0 

0 

0 

8.32 

0 

0 

0 

0 

0 

0/B 

0 

0 

0 

8.33 

0 

0 

0 

0 

0 

0/B 

0 

0 

0     1 

8.34 

0 

0 

0 

0 

0 

0/B 

0 

0 

0 

8.36 

0.333/D 

0/D 

0/D 

0/D 

0 

0 

0 

0.143/D 

0.091 

8.37 

0.600/D 

0/D 

0/D 

0/D 

0 

0 

0 

0.333/D 

0.143 

8.38 

0.600 

0 

0 

0 

0 

0/B 

0 

0.333 

0.143 

8.40 

0.500 

0 

0 

0 

0 

0/B 

0 

0.250 

0.125 

8.41 

0.500 

0 

0 

0 

0 

0/B 

0 

0.333 

0.143 

8.42 

0 

0 

0.500 

0.143 

1.00 

0.500/B 

0.333 

0 

0 

8.44 

0.333 

0 

0 

0 

0 

0/B 

0 

0.167 

0.100 

8.45 

0.200 

1.00 

0.333 

0.125 

0.500 

0.333/B 

0.250 

0 

0 

8.46 

0.333 

0 

0 

0 

0 

0/B 

0.333 

0.333 

0.200 

8.48 

0.500 

0 

0.500 

0.143 

1.00 

0.500/B 

0 

0 

0 

1    8.50 

0 

0 

0 

0 

0 

0/B 

0 

0 

« 
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TABLE  C.8: 

JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 
VERSION  VIM 

1 

8.11 

8.12 

8.13 

8.14 

8.15 

8.18 

8.19 

8.23 

8.25 

8.26 

8.27 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0.500/B 

0.500/B 

8.28 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0.667/B 

0.667/B 

8.29 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0.400/B 

0.400/B 

8.30 

0/^ 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.31 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.32 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.33 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.34 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.36 

0 

0 

0.250 

0.333/ 
D 

0.571 

0.100 

0.300 

0 

0 

0 

8.37 

0 

0 

0.50 

0.222/ 
D 

1.00 

0.167 

0.333 

0 

0 

0 

8.38 

0/B 

0/B 

0.500/ 
B 

0.222 

l.OO/B 

0.167/ 
B 

0.33/ 
B 

0/B 

0/B 

0/B 

8.40 

0/B 

0/B 

0.400/ 
B 

0.30 
0 

1.00/B 

0.143/ 
B 

0.429 
/B 

0/B 

0/B 

0/B 

8.41 

0/B 

0/B 

0.500/ 
B 

0.33 
3 

0/B 

0.167/ 
B 

0.500 
/B 

0/B 

0/B 

0/B 

8.42 

0.333/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0.500 
/B 

0/B 

0/B 

8.44 

0/B 

0/B 

0.286/ 
B 

0.25 
0 

0.667/ 
B 

0.111/ 
B 

0.429 
/B 

0/B 

0/B 

0/B 

8.45 

0.250/B 

0/B 

0/B 

0.125 

0.500/ 
B 

0.250/ 
B 

0/B 

0.33/ 
B 

0/B 

0/B 

8.46 

0/B 

0/B 

0.500/ 
B 

0.333 

1.00/B 

0/B 

0.500 
/B 

0/B 

0/B 

0/B 

8.48 

0.333/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0.500 
/B 

0/B 

0/B 

8.50 

0/B 

0/B 

0/B 

0 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 
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TABLE  C.8:   JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vm 


8.27 

8.28 

8.29 

8.30 

8.31 

8.32 

8.33 

8.27 

B 

- 

- 

- 

- 

- 

- 

8.28 

0.500/B 

B 

- 

- 

- 

- 

- 

8.29 

0.333/B 

0.400/B 

B 

- 

- 

- 

- 

8.30 

0/B 

0/B 

0/B 

B 

- 

- 

- 

8.31 

0/B 

0/B 

0/B 

0.444/B 

B 

- 

- 

8.32 

0/B 

0/B 

0/B 

0.667/B 

0.500/B 

B 

- 

8.33 

0/B 

0/B 

0/B 

0.667/B 

0.500/B 

0.800/B 

B 

8.34 

0/B 

0/B 

0/B 

0.364/B 

0.308/B 

0.400/B 

0.400/B 

8.36 

0 

0 

0 

0 

0 

0 

0 

8.37 

0 

0 

0 

0 

0 

0 

0 

8.38 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.40 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.41 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.42 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.44 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.45 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.46 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.48 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

8.50 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 

0/B 
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TABLE  C.8:   JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  Vffl 


8.34 

8.36 

8.37 

8.38 

8.40 

8.41 

8.42 

8.44 

8.45 

8.46 

8.48 

8.5 
0 

8.3 
4 

B 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.3 
6 

0/B 

D 

- 

- 

- 

- 

- 

- 

- 

- 

8.3 

7 

0 

0.429 
/D 

D 

- 

- 

- 

- 

- 

- 

- 

8.3 
8 

0/B 

0.429 

1.00 

B 

- 

- 

- 

- 

- 

- 

8.4 
0 

0/B 

0.500 

0.750 

0.750 
/B 

B 

- 

- 

- 

- 

- 

8.4 
1 

0/B 

0.444 

0.600 

0.600 
/B 

0.833 
/B 

B 

- 

- 

- 

- 

8.4 

2 

0/B 

0 

0 

0/B 

0/B 

0/ 
B 

B 

- 

- 

- 

- 

8.4 
4 

0/B 

0.364 

0.375 

0.375 
/B 

0.444 
/B 

0.500 
/B 

0/ 
B 

B 

- 

- 

- 

- 

8.4 

5 

0/B 

0.125 

0.250 

0.250 
/B 

0.200 
/B 

0.250 
/B 

1.000 
/B 

0.143 
/B 

B 

- 

- 

- 

8.4 
6 

0/B 

0.333 

0.400 

0.400 
/B 

0.500 
/B 

0.600 
/B 

0/ 
B 

0.375 
/B 

0.667 
/B 

B 

- 

- 

8.4 
8 

0/B 

0 

0 

0/B 

0/B 

0/ 
B 

0/ 
B 

0/ 
B 

1.00/ 
B 

0/ 
B 

B 

- 

8.5 
0 

0.083 
/B 

0 

0 

0/B 

0/B 

0/ 
B 

0/ 
B 

0/ 
B 

0/ 
B 

0/ 
B 

0/ 
B 

B 
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APPENDIX  D:  FUNCTIONAL  FAULT  TYPE  DISTRIBUTION 

In  this  Appendix  we  present  (Tables  D.l  to  D.8)  the  relative  frequency 
of  occurrence  of  functional  fault  types  in  the  eight  versions  of  CONFLICT 
(Shimeall  1991b).  The  fault  types  are  classified  in  types  O,  I,  n,  m,  IV,  V 
and  VI  according  to  the  discussion  in  Section  B.7  of  Chapter  m  . 

Some  of  the  faults  included  in  the  failure  region  library  by  Shimeall 
(1991a)  do  not  have  a  specific  location  and,  therefore,  cannot  be  assigned  to 
any  specific  program  module.  However,  it  is  possible  by  their  description  to 
detennine  which  part  of  the  CONFLICT  function  each  of  them  affects.  These 
faults  occupy  the  generic  entries  (labeled  as  Others,  movement,.,  etc)  in  bold 
type  that  precede  each  functional  group  of  program  modules. 

Faults  that  have  been  assigned  to  multiple  functional  groups  contribute 
to  the  fault  count  of  each  group  by  an  appropriate  fraction  of  fault.  For 
example,  a  fault  distributed  in  three  groups,  will  contribute  to  the  fault  count 
of  each  group  by  1/3. 

The  observed  fault  type  frequencies  are  compared  with  the  expected 
frequencies  of  a  simple  model  that  assumes  uniform  distribution  of  faults  as 
in  Appendix  B. 
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TABLE  D.l  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  I^" 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Otiiers 

0 

561 

- 

8/26 

6.2 

0.24 

Conflict 

0 

94 

- 

- 

- 

- 

Ceiling 

0 

18 

- 

- 

- 

- 

Mini 

0 

7 

- 

- 

- 

- 

MinR 

O 

7 

- 

- 

- 

- 

MaxIR 

O 

7 

- 

- 

- 

- 

Outside  Range 

O 

17 

- 

- 

- 

- 

Verify  Input 

0 

20 

1.17 

- 

- 

- 

Check  Params 

0 

32 

1.7.  1.6 

- 

- 

- 

Check  Anny 
Values 

0 

115 

1.10,  1.11 

- 

- 

■ 

Prepare  Output 

0 

49 

1.23,  1.1 

- 

- 

- 

Initialization 

0 

114 

- 

- 

- 

Update  Aimy 
Values 

0 

24 

- 

- 

- 

" 

Update 

0 

71 

1.2 

- 

- 

- 

10 


The  fault  rate  is  26/2352  =  0.01105   faults  per  line  (including  data 
declarations  in  the  line  count) 
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TABLE  D.l  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  I 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Movement 

I 

160 

- 

1/26 

1.77 

0.068 

BatVelocV 

I 

23 

- 

- 

- 

- 

AltitudeZ 

I 

30 

- 

- 

- 

- 

DistD 

I 

4 

- 

- 

- 

- 

Position 

I 

72 

1.8 

- 

- 

- 

Movement 

I 

31 

- 

- 

- 

- 

Observation 

II 

240 

- 

3/26 

2.65 

0.10 

SizeListLoc 

II 

14 

- 

- 

- 

- 

HeightH 

II 

20 

- 

- 

- 

- 

FindAngle 

II 

21 

- 

- 

- 

- 

First  Condi- 
tion 

II 

41 

- 

- 

- 

Second 
Condition 

II 

25 

- 

- 

- 

- 

Visual 
Contrast 

II 

10 

- 

- 

- 

" 

1    ObservJam 

II 

18 

1.27 

- 

- 

- 

Third 
1     Condition 

II 

27 

- 

- 

- 

1  Observation 

II 

64 

1.14,  1.26 

- 

- 

- 
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TABLE  D.l  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  I 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Attrition 

III 

355 

- 

4/26 

3.92 

0.151 

Attrition 

III 

10 

1.13 

- 

- 

- 

Set  Fire 

Upon 

Coords 

III 

44 

1.13,  1.16.  1.15 

" 

- 

- 

Assign  LL 
Coords 

ni 

21 

- 

- 

- 

- 

KiUedK 

III 

46 

- 

- 

- 

- 

Num  Killers 

NK 

III 

28 

- 

- 

- 

- 

Killere 
AvailKA 

III 

39 

- 

- 

- 

- 

Times 

Killers 

UsedKU 

III 

40 

- 

- 

- 

- 

Total  Weap 
In  Use  NW 

III 

51 

1.20 

- 

- 

- 

ClearDead 
Squads 

in 

30 

- 

- 

- 

- 

For  Each 
Weap 

III 

46 

- 

- 

- 

- 

Communica 
tion 

IV 

678 

- 

3/26 

7.49 

0.288 

Check 
ComMsg 

IV 

21 

- 

- 

- 

- 

Send 
Reports 

IV 

47 

- 

- 

- 
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TABLE  D.l  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  I 


1 '' 

Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Communica 
tion 

IV 

10 

- 

- 

- 

■ 

Total 

Squads 

SendingNS 

IV 

22 

- 

- 

- 

TotalSquads 
ReceivNR 

IV 

22 

- 

- 

- 

- 

TotalSquads 
Jamming 

IV 

22 

- 

- 

- 

- 

TotalSquads 

Processing 

NR 

IV 

22 

- 

- 

- 

PuantoList 

IV 

42 

1.22 

- 

- 

- 

SendMsgs 

IV 

121 

- 

- 

- 

- 

Process 
Command 
Messages 

IV 

29 

- 

- 

- 

Process 
Report 

Messages 

IV 

69 

~ 

- 

- 

Msg 
Receipt 
DelayRd 

IV 

52 

~ 

- 

- 

ManipProce 
ss  List 

rv 

78 

- 

- 

- 

" 

ManipMsg 
Queue 

IV 

33 

- 

- 

- 

■ 
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TABLE  D.l  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  I 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

PutMsg  On 
Sent  LL 

IV 

22 

- 

- 

- 

- 

Instantiate 
CommandMsg 

IV 

66 

1.25,  1.19 

- 

- 

- 

Environment 

V 

143 

- 

4/26 

1.58 

0.061 

Chech  Weather 

V 

26 

1.3,  1.4 

- 

- 

- 

TeirMove  TM 

V 

25 

1.18,  1.12 

- 

- 

- 

Weather 
SevFactor  WF 

V 

22 

- 

- 

- 

- 

Weather 
ObservWO 

V 

12 

- 

- 

- 

- 

Weather 
MoveWM 

V 

12 

- 

- 

- 

- 

Slope  Intensity 
IS 

V 

24 

- 

- 

- 

- 

Altitude 
Intensity  lA 

V 

7 

- 

- 

- 

- 

IntensityLocIL 

V 

6 

- 

- 

- 

- 

Location 
Intensity  BI 

V 

9 

- 

- 

- 

- 
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TABLE  D.l  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  I 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Restoration 

VI 

215 

- 

3/26 

2.38 

0.092 

Squad  Alive 

VI 

4 

1.9 

- 

- 

- 

Bat  Alive 

VI 

16 

- 

- 

- 

- 

Restoration 

VI 

8 

- 

- 

- 

- 

TotalRestore 

dCasualtiesF 

F 

VI 

36 

■ 

- 

- 

- 

Coefficient 

VI 

28 

- 

- 

- 

- 

MunSquads 
Restoring 

NF 

VI 

19 

" 

- 

Restore 
SuppAmtFS 

VI 

23 

- 

- 

- 

Restore 
FactoiF 

VI 

54 

1.5,  1.28 

- 

- 

CalcEnduiE 

VI 

27 

- 

- 

- 
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TABLE  D.2  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION    n" 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Others 

O 

184 

- 

0/26 

3.11 

0.119 

Conflict 

0 

96 

- 

- 

- 

- 

MinReal 

0 

7 

- 

- 

- 

- 

Minlnt 

o 

7 

- 

- 

- 

- 

Max 

0 

7 

- 

- 

- 

Maxint 

0 

7 

- 

- 

- 

- 

Roof 

o 

11 

- 

- 

- 

- 

Floor 

0 

9 

- 

- 

- 

- 

Output 

o 

5 

- 

- 

- 

- 

Simulation 

0 

17 

- 

- 

- 

- 

Initializatio 
n 

0 

18 

- 

- 

- 

- 

Movement 

243 

2.10,  2.11.  2.13 

3/26 

4.10 

0.158 

TMove 

25 

- 

- 

- 

- 

ScaleSqua 
d 

67 

- 

- 

- 

- 

Positioning 

16 

- 

- 

- 

- 

Velocity 

21 

- 

- 

- 

11 


The  fault  rate  is  26/1540  =  0.0169  faults  per  line  (including  data 
declarations  in  the  line  count) 
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TABLE  D.2  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  n 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Xmove 

I 

9 

- 

- 

- 

- 

Ymove 

I 

9 

- 

- 

- 

- 

Movement 

I 

37 

- 

- 

- 

- 

Poslnit 

I 

14 

- 

- 

- 

- 

Movelnit 

I 

11 

- 

- 

. 

MoveOut 

I 

34 

- 

- 

- 

- 

Observation 

II 

227 

2.15,  2.26 

5/26 

3.832 

0.147 

Angle  Big 
Enough 

II 

60 

2.7,  2.8 

- 

- 

- 

Slope 

II 

4 

- 

- 

- 

- 

Can  J  Seek 

II 

9 

- 

- 

- 

- 

Observation 

n 

50 

- 

- 

- 

Calc 
Contrast 

II 

24 

- 

- 

- 

- 

FindPt 

II 

5 

2.12 

- 

- 

- 

No 
Obstacles 

II 

20 

- 

- 

- 

- 

NotObs 
Jammed 

II 

20 

- 

- 

- 

- 

OJamming 

II 

25 

- 

- 

- 

- 

Obslnit 

II 

10 

- 

- 

- 

- 
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TABLE  D.  2  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES 

IN  VERSION  n 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Attrition 

III 

287 

- 

0/26 

4.85 

0.186 

Attrition 

III 

5 

- 

- 

- 

- 

Attritlnflict 

III 

9 

- 

- 

- 

- 

Weapons 

III 

63 

- 

- 

- 

- 

FiieCoord 

III 

56 

- 

- 

- 

- 

Suffering 

III 

94 

- 

- 

- 

- 

Attritlnit 

III 

35 

- 

- 

- 

- 

AttritOut 

III 

25 

- 

- 

- 

- 

Communic 
ation 

IV 

369 

2.16,2.17,  2.18, 

2.19,  2.20, 
2.21,  2.22,  2.23 

14/26 

6.23 

0.240 

Communic 
ation 

IV 

9 

14,  2.25 

- 

- 

- 

Update 
Comm 

IV 

59 

- 

- 

- 

- 

AddToQ 

IV 

21 

- 

- 

- 

- 

Create 
Reports 

IV 

41 

2.1 

- 

- 

- 

Create 
Commands 

IV 

28 

2.2 

- 

- 

- 

PuU 
FromQ 

IV 

32 

- 

- 

- 

- 

Relay 

Messages 

IV 

55 

2.3 

- 

- 

- 

Consume 
Reports 

IV 

58 

- 

- 

- 

- 
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TABLE  D.  2  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES 

IN  VERSION  n 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Consume 
Commands 

IV 

51 

2.5 

- 

- 

- 

Commlnit 

IV 

15 

- 

- 

- 

- 

Environme 
nt 

V 

141 

- 

3/26 

2.38 

0.092 

Dist 

V 

4 

- 

- 

- 

- 

Alt 

V 

25 

2.30,  2.6,  2.31 

- 

- 

- 

Poslntens 

V 

31 

- 

- 

- 

- 

WTotal 

V 

27 

- 

- 

- 

- 

WMove 

V 

21 

- 

- 

- 

- 

WObs 

V 

21 

- 

- 

- 

- 

Height 

V 

12 

- 

- 

- 

- 

Restoration 

VI 

89 

2.14 

1/26 

1.50 

0.058 

Restoration 



VI 

89 

- 

- 

- 

- 
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TABLE  D.3  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  ra'^ 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Others 

0 

312 

- 

7.5/40 

10.4 

0.26 

Conflict 

0 

81 

3.10,  3.17, 
3.21,  3.32 

- 

- 

- 

max 

0 

5 

- 

- 

- 

- 

min 

0 

4 

- 

- 

- 

- 

Cieling 

0 

9 

- 

- 

- 

- 

Change 

o 

6 

- 

- 

- 

- 

Output 

0 

64 

- 

- 

- 

- 

DataUpdate 

o 

75 

3.15 

- 

- 

- 

Perfonn 

Passive 

Functions 

o 

5 

- 

- 

- 

- 

InitVals 

0 

63 

3.5.  3.24,  3.26 

- 

- 

- 

Movement 

I 

94 

- 

2/40 

3.13 

0.078 

BatPosition 

I 

67 

3.25,  3.6 

- 

- 

- 

positioning 

I 

8 

- 

- 

- 

- 

1         Move 

I 

19 

- 

- 

- 

- 

12 


The  fault  rate  is  40/1200  =  0.00333   faults  per  line  (including  data 
declarations  in  the  line  count) 


112 


TABLE  D.3  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  m 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Observation 

II 

147 

3.34 

3/40 

4.9 

0.123 

SubAngle 

II 

63 

3.13 

- 

- 

- 

1      sighting 

n 

66 

3.43 

- 

- 

- 

Summation 

II 

13 

- 

- 

- 

- 

observe 

II 

5 

- 

- 

- 

- 

Attrition 

III 

131 

3.7,  3.8 

7/40 

4.37 

0.109 

DoDamage 

III 

9 

- 

- 

- 

- 

Weapon 
Sighting 

III 

57 

3.9,  3.18,  3.19, 
3.44,  3.45 

- 

- 

- 

Attrition 

III 

40 

- 

- 

- 

- 

Aggression 

III 

19 

- 

- 

- 

- 

DoAction 

III 

6 

- 

- 

- 

- 

Communica 
tion 

IV 

407 

3.36,  3.37, 
3.38,  3.39, 
3.40,  3.41, 
3.42,  3.35 

13.5/40 

13.57 

0.339 

InitRec 

IV 

14 

- 

- 

- 

- 

ScanQueue 

IV 

26 

- 

- 

- 

- 

PuHnQueue 

IV 

16 

3.2 

- 

- 

- 

Follow 
Command 
Messages 

IV 

132 

3.3,  3.4 
3.15 

- 

- 

- 

ScanQueue 

IV 

22 

- 

- 

- 

- 
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TABLE  D.3  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  in 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Receive 
Messages 

IV 

58 

3.23 

- 

- 

- 

NM 

IV 

19 

3.1 

- 

- 

- 

Compare  Reed 
Messages 

IV 

46 

- 

- 

- 

- 

Send 
Observations 

IV 

31 

- 

- 

- 

Send  Orders 

IV 

28 

- 

- 

- 

- 

Send  Messages 

rv 

5 

- 

- 

- 

- 

Update 

IV 

10 

- 

- 

- 

- 

Environment 

V 

85 

- 

5/40 

2.833 

0.071       1 

Distance 

V 

4 

- 

- 

- 

- 

findA 

V 

16 

3.11,  3.20, 
3.28,  3.16 

- 

- 

- 

Altitude 

V 

6 

- 

- 

- 

- 

BI 

V 

15 

- 

- 

- 

- 

TM 

V 

12 

3.12 

- 

- 

- 

WF 

V 

14 

- 

- 

- 

- 

WM 

V 

10 

- 

- 

- 

- 

WO 

V 

8 

- 

- 

- 

- 

Restoration 

VI 

24 

3.33 

2/40 

0.800 

0.020 

Restore 

IV 

24 

3.22 

- 

- 

- 
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TABLE  D.4  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  IV'^ 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Others 

0 

271 

- 

0/23 

3.124 

0.136 

Conflict 

0 

128 

- 

- 

- 

- 

Initialize 

0 

88 

- 

- 

- 

- 

Prep  Output 

0 

55 

- 

- 

- 

- 

Movement 

I 

209 

- 

0/23 

2.410 

0.105 

Movement 

I 

38 

- 

- 

- 

- 

CalcVelocity 

I 

23 

- 

- 

- 

- 

MTeirain 

I 

28 

- 

- 

- 

- 

Position  Squads 

I 

120 

- 

- 

- 

- 

Observation 

II 

266 

- 

11.5/23 

3.067 

0.133 

Observation 

n 

49 

4.6,  4.7, 

4.10,  4.12, 

4.18 

- 

- 

- 

Init  Visual 

II 

16 

4.5 

- 

- 

- 

ValidObservatio 
n 

II 

13 

4.20 

- 

- 

- 

Angle 

II 

64 

- 

- 

- 

. 

Line 

II 

24 

4.9, 
4.20,4.22 

- 

- 

- 

Obs  Contrast 

II 

25 

- 

- 

- 

" 

Location  List 

II 

20 

4.9 

- 

- 

Visual  Contrast 

II 

30 

- 

- 

- 

- 

13 


The  fault  rate  is  23/1995=0.01153  faults  per  line  (including  data 
declarations  in  the  line  count) 
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TABLE  D.4  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  rv 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

ObsJamming 

II 

25 

4.44.16, 
4.20 

- 

- 

- 

Attrition 

III 

318 

- 

2.5/23 

3.67 

0.159 

Attrition 

III 

3 

- 

- 

- 

- 

Inflict 

III 

12 

- 

- 

- 

- 

SetCoondinates 

III 

55 

4.8,  4.19 

- 

- 

- 

Weapon  Count 

in 

21 

- 

- 

- 

- 

Weapon  Inflict 

III 

90 

- 

- 

- 

- 

Suffer 

III 

13 

- 

- 

- 

- 

Init  Fire  List 

III 

60 

- 

- 

- 

- 

Squad  Damage 

III 

30 

4.3 

- 

- 

- 

Is  Destroyed 

III 

4 

- 

- 

- 

- 

Is  Casualty 

III 

30 

- 

- 

- 

- 

Communicatio 
ns 

IV 

584 

- 

7/23 

6.733 

0.292 

Communicatio 
ns 

IV 

91 

- 

- 

- 

- 

Process  Msg 

IV 

65 

4.13,  4.14, 
4.26 

- 

- 

- 

SendMsg 

IV 

54 

- 

- 

- 

- 

QueueMsg 

IV 

90 

4.1 

- 

- 

- 

ReceiveMsg 

IV 

81 

- 

- 

- 

- 

AddToList 

IV 

50 

4.15 

- 

- 

- 

CmdReplace 

IV 

74 

4.11,4.24 

- 

- 

- 
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TABLE  D.4  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  IV 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Report 
Messages 

IV 

45 

- 

- 

- 

- 

Command 
Messages 

IV 

34 

- 

- 

- 

- 

Environment 

V 

144 

- 

1/23 

1.66 

0.072 

Altitude 

V 

23 

4.2 

- 

- 

- 

Distance 

V 

39 

- 

- 

- 

- 

WE 
Observation 

V 

22 

- 

- 

- 

- 

WE  Movement 

V 

21 

- 

- 

- 

- 

Slope  Intensity 

V 

17 

- 

- 

- 

- 

Alt  Intensity 

V 

22 

- 

- 

- 

- 

Restoration 

VI 

203 

- 

1/23 

2.340 

0.102 

Restoration 

VI 

36 

- 

- 

- 

- 

New  Casualties 

VI 

14 

- 

- 

- 

_ 

Total 
Casualties 

VI 

11 

- 

- 

- 

Restore 
Amount 

VI 

25 

- 

- 

- 

- 

Restore 
Supplies 

VI 

18 

- 

- 

- 

" 

SquadFixers 

VI 

8 

- 

- 

- 

- 

Battalion 
Size 

VI 

19 

4.21 

- 

- 

" 

Transfer 

VI 

72 

- 

- 

- 

- 
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TABLE  D.5  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  V'^ 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Others 

0 

362 

- 

13/40 

9.378 

0.234 

Conflict 

O 

81 

- 

- 

- 

- 

Simulate 

0 

31 

- 

- 

- 

- 

Validate 

0 

149 

5.6,  5.7,  5.9, 
5.10.  5.11, 
5.12,  5.28, 
5.30,  5.32 

- 

- 

- 

Set  Initial 
Values 

o 

45 

5.3,  5.14,  5.31 

- 

- 

- 

Output 
Results 

0 

56 

5.42 

- 

- 

- 

Movement 

I 

102 

- 

0/40 

2.642 

0.066 

Position 

I 

58 

- 

- 

- 

- 

Movement 

I 

44 

- 

- 

- 

- 

Observation 

n 

305 

5.20,  5.43,  5.37 

6/40 

7.902 

0.198 

Observation 

II 

237 

- 

- 

- 

- 

SpacePoints 

II 

50 

5.1,  5.17 

- 

- 

- 

IntnstyLoc 

II 

18 

5.13,  5.41 

- 

- 

- 

Attrition 

III 

280 

- 

2/40 

7.254 

0.181 

Change  Old 

III 

10 

- 

- 

- 

- 

1      Change 
1       Squad 

III 

9 

- 

- 

- 

Attrition 

III 

11 

- 

- 

- 

Suffer 

III 

60 

- 

- 

- 

- 

14 


The  fault  rate  is  40/1544  =0.026  faults  per  line  (including  data 
declarations  in  the  line  count) 
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TABLE  D.5  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  V 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Inflict 

m 

152 

5.15,  5.16 

- 

- 

- 

Wear 

in 

38 

- 

- 

- 

Com 
munication 

IV 

349 

5.2,  5.19, 
5.24,5.25,5.2 
6,  5.27,5.39, 

5.40 

5.21,5.22,5.2 

3 

15.5/40 

9.041 

0.226 

Comnicat 

IV 

239 

5.4,  5.5,  5.8. 
5.33,  5.34 

- 

- 

- 

Store  Mess 

IV 

39 

- 

- 

- 

ReprtMess 

IV 

25 

- 

- 

- 

- 

Commnd 
Mess 

IV 

27 

- 

- 

- 

- 

Update 
CommVars 

IV 

19 

- 

- 

- 

- 

Environment 

V 

88 

- 

1.5/40 

2.280 

0.057 

Distance 

V 

7 

- 

- 

- 

- 

Altitude 

V 

19 

5.13,  5.41 

- 

- 

- 

CompWeath 

V 

23 

- 

- 

- 

- 

WEffMov 

V 

12 

- 

- 

- 

- 

I'bffMov 

V 

19 

- 

- 

- 

- 

WEffObs 

V 

8 

- 

- 

- 

- 

Restoration 

VI 

59 

5.18 

2/40 

1.528 

0.038 

Restoration 

VI 

59 

5.35 

- 

- 

- 
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TABLE  D.6  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  VI^^ 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 

Frequen 

cy 

Others 

0 

409 

4.5/20 

3.686 

0.184 

Conflict 

0 

95 

6.2 

- 

- 

- 

Min 

0 

9 

- 

- 

- 

- 

Max 

0 

9 

- 

- 

- 

- 

IMin 

O 

9 

- 

- 

- 

- 

IMax 

0 

9 

- 

- 

- 

- 

Ceiling 

0 

14 

- 

- 

- 

- 

CheckBatt 
Constar^s 

0 

47 

- 

- 

- 

- 

Init  Battalion 

88 

6.8,  6.20 

- 

- 

- 

Initialize 

0 

44 

- 

- 

- 

- 

Peifom 
Simulation 

0 

24 

6.2,  6.4 

- 

- 

- 

Perfomi  OneDt 

0 

12 

- 

- 

- 

- 

Prepare  For 
NextDt 

O 

12 

6.7 

- 

- 

- 

Deteimine 
Output 

O 

37 

- 

- 

- 

- 

15 


The  faiilt  ra     is  20/2219=0.0090   faults  per  line  (including  data 
declarations  ^^jl  the  line  count) 
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TABLE  D.6  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  VI 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 

Frequen 

cy 

Movement 

I 

96 

- 

0/20 

0.865 

0.043 

Update 
Battalion 
Velocity 

I 

21 

- 

- 

- 

- 

AlignSquads 

I 

49 

- 

- 

- 

- 

Movement 

I 

9 

- 

- 

- 

- 

Move 
Battalion 

I 

17 

- 

- 

- 

■ 

Observation 

II 

323 

- 

3/20 

2.911 

0.146 

Create 
LOSList 

11 

11 

- 

- 

- 

■ 

Observation 

II 

12 

- 

- 

- 

- 

1    GenObsList 

II 

42 

- 

- 

- 

- 

Observable 

II 

16 

6.1 

- 

- 

- 

AngleSub 
Greater 

II 

79 

- 

- 

- 

- 

Updat 
LOSList 

II 

13 

6.3 

- 

- 

- 

LOSClear 

II 

24 

6.16 

- 

- 

- 

CntrstOk 

II 

30 

- 

- 

- 

- 

Location 
Intensity 

II 

30 

- 

- 

- 

- 

Obs  Jamming 

II 

28 

- 

- 

- 

- 

SumObs 
ToNextBatt 

IV 

38 

- 

- 

- 

- 
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TABLE  D.6  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  VI 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Attrition 

in 

201 

6.23 

5/20 

1.812 

0.091 

Attrition 

III 

12 

6.17 

- 

- 

- 

Num 
Weapons 

m 

27 

6.9 

- 

- 

- 

Track 
Weapons 

III 

21 

6.5 

- 

- 

- 

UpdatUse 
List 

m 

29 

- 

- 

- 

- 

Choose 
Targets 

III 

45 

6.6 

- 

- 

- 

Suffer 
Attrition 

III 

67 

- 

- 

- 

- 

Communic 
ations 

IV 

900 

6.24 

7.5/20 

8.112 

0.406 

Include 
CommObs 

IV 

39 

6.18 

- 

- 

- 

Collect 

Finished 

Report 

IV 

41 

- 

- 

- 

- 

UpdateLL 

IV 

42 

6.14 

- 

- 

- 

Communic 
ation 

IV 

16 

- 

- 

- 

- 

Send 

Communic 

ations 

IV 

35 

- 

- 

- 

- 

Send 
Report 

IV 

35 

6.2 

- 

- 

- 
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TABLE  D.6  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  VI 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 

Frequen 

cy 

NewNum 
Send 

IV 

28 

- 

- 

- 

" 

Send 
Command 

IV 

29 

- 

- 

- 

ReceiveCom- 
munications 

IV 

11 

- 

- 

- 

- 

Find  Receiv- 
ing Delay 

IV 

55 

6.12 

- 

- 

- 

Receive 
Reports 

IV 

56 

- 

- 

- 

- 

Receive 
Commands 

IV 

57 

- 

- 

- 

- 

Update 
NimiVars 

IV 

30 

- 

- 

- 

- 

ProcessCom- 
munications 

IV 

11 

- 

- 

- 

Handle 
Queuing 

IV 

8 

- 

- 

- 

■ 

Queue 
Reports 

IV 

38 

- 

- 

- 

Find  Queue 
Report 

IV 

19 

- 

- 

- 

- 

Queue 
Commands 

IV 

38 

- 

- 

- 

- 

Find  Queue 
Spot 

IV 

19 

- 

- 

- 

- 

Processing 
Delay 

IV 

24 

- 

- 

- 

- 
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TABLE  D.6  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  VI 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 

Frequen 

cy 

Process 
Messages 

IV 

67 

- 

- 

- 

- 

Find  Next 
Report 

IV 

19 

- 

- 

- 

- 

Find  Next 
Command 

IV 

20 

- 

- 

- 

- 

Take  A 
Command 

IV 

13 

- 

- 

- 

- 

Take  A 
Report 

IV 

13 

- 

- 

- 

- 

New  Num 
Processing 

IV 

20 

- 

- 

- 

_ 

Collect 
Commands 

IV 

42 

6.2,  6.11 

- 

- 

- 

Collect 
Command 

IV 

43 

- 

- 

Pufln 
Commands 

IV 

32 

6.13,  6.15 

- 

- 

Environment 

V 

176 

- 

0/20 

1.586 

0.079 

Distance 

V 

9 

- 

- 

- 

-         1 
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TABLE  D.6  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  VI 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Height 

V 

28 

- 

- 

- 

- 

Update 
Weather 

V 

7 

- 

- 

- 

- 

Update 
Present 
Events 

V 

41 

- 

- 

- 

- 

Add  New 
Events 

V 

24 

- 

- 

- 

- 

Weather 
Severity 

V 

42 

- 

- 

- 

- 

WEOn 
Movement 

V 

25 

- 

- 

- 

- 

Restoration 

VI 

114 

- 

0/20 

1.027 

0.051 

Restoration 

VI 

17 

- 

- 

- 

- 

NewNum 
Fixers 

VI 

24 

- 

- 

- 

- 

Apportion 
Fixing 

VI 

53 

- 

- 

- 

- 

Remove 

Destroyed 

Squads 

VI 

20 

- 

- 

- 

- 
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TABLE  D.7  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  vn^^  ^^ 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequenc 

y 

Expected 
Faults 

Relative 

Frequen 

cy 

Others 

0 

603 

- 

4.5/31 

10.385 

0.335 

Conflict 

0 

36 

7.28 

- 

- 

- 

float 

0 

4 

- 

- 

- 

- 

WriteError 

0 

20 

- 

- 

- 

- 

Process 

0 

107 

- 

- 

- 

- 

Check  Params 

0 

34 

- 

- 

- 

- 

Check  NAnny 

0 

13 

- 

- 

- 

- 

CheckPer 
Battalion 

o 

97 

7.6,  7.7,  7.8, 
7.14 

- 

- 

- 

CheckPeiSquad 

0 

21 

- 

- 

- 

- 

CheckPer  Enemy 

0 

15 

- 

- 

- 

- 

Check  Per 
Weapon 

0 

32 

- 

- 

- 

- 

Makeint 

0 

7 

- 

- 

- 

- 

Initialize 

0 

54 

7.28 

- 

- 

- 

SetSuad 

0 

28 

- 

- 

- 

- 

Update  Info 

0 

37 

- 

- 

- 

- 

Update  Battalion 

0 

42 

7.28 

- 

- 

- 

16 


The  fault  rate  is  31/1800=0.0172   faults  per  line  (including  data 
declarations  in  the  line  count) 


17 


In  this  Version,  certain  sub  procedures  are  declared  in  more  than  one 
locations 
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TABLE  D7.  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  vn 


I 

Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Min 

O 

6 

- 

- 

- 

- 

Min 

0 

5 

- 

- 

- 

- 

SetOutput 

0 

24 

- 

- 

- 

- 

Greatest 

0 

6 

- 

- 

- 

■ 

Least 

0 

6 

- 

- 

- 

CheckBatta 
lionlnfo 

0 

9 

- 

- 

- 

- 

Movement 

289 

- 

0/31 

4.977 

0.161 

Invalid 
Position 

10 

- 

- 

- 

- 

Velocity 

20 

- 

- 

- 

- 

SetSquad 

28 

- 

- 

- 

- 

Position 
Squadrons 

19 

- 

- 

- 

- 

1  SetPosition 

48 

- 

- 

- 

- 

Velocity 

20 

- 

- 

- 

- 

SetPosition 

48 

- 

- 

- 

- 

Update 
Position 

24 

- 

- 

- 

- 

Movement 

19 

- 

- 

- 

- 

Velocity 

20 

- 

- 

- 

- 

Get 
Difference 

33 

- 

- 

- 

- 
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TABLE  D.7  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  vn 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Il 

Relative 

Frequen 
cy 

Observation 

II 

268 

7.17,  7.35 

7/31 

4.616 

0.149 

Observation 

II 

32 

7.5.  7.10,  7.13 

- 

- 

- 

Visible  Squad 

II 

40 

7.1 

- 

- 

- 

SubAngle 

II 

27 

- 

- 

- 

- 

GetAngle 

II 

30 

- 

- 

- 

- 

Series 

II 

17 

7.32 

- 

- 

- 

QearView 

II 

31 

- 

- 

- 

- 

OContrast 

II 

35 

- 

- 

- 

- 

Intensity 

II 

18 

- 

- 

- 

- 

OJamming 

II 

19 

- 

- 

- 

- 

Calc  Num 
Observ 

II 

19 

- 

- 

- 

- 

Attrition 

III 

221 

- 

5/31 

3.806 

0.123 

Initialize 
WeapData 

III 

15 

7.15 

- 

- 

- 

Initialize 
WeapData 

III 

15 

- 

- 

- 

- 

Attrition 

III 

13 

- 

- 

- 

- 

Inflict 
Attrition 

III 

45 

7.2.  7.3.  7.29 

- 

- 

' 

SetFire 

III 

28 

- 

- 

- 

- 
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TABLE  D7.  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  vn 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

CalcNum 
WeapToUse 

III 

48 

7.27 

- 

- 

- 

Calculate 
Damages 

III 

31 

- 

- 

- 

- 

Set  Damage 

III 

9 

- 

- 

- 

- 

Get  Status 

III 

17 

- 

- 

- 

- 

Com 
munication 

IV 

181 

7.18.  7.9. 

7.19, 
7.20,7.21. 
7.22,  7.23, 
7.24,7.25 

10.5/31 

3.117 

0.101 

Check 

Messages 

IV 

21 

- 

- 

- 

- 

Command 
Mess 

IV 

80 

7.4 

- 

- 

- 

RecDelay 

IV 

14 

- 

- 

- 

- 

Jammed 
Squads 

IV 

26 

- 

- 

- 

- 

Makeint 

IV 

7 

- 

- 

- 

- 

Incoiporate 

IV 

33 

7.28 

- 

- 

- 

Environment 

V 

141 

- 

2/31 

2.428 

0.078 

Check 
Weather 

V 

19 

- 

- 

- 

- 

GetTs 

V 

7 

7.11.  7.33 

- 

- 

- 

Altitude 

V 

18 

- 

- 

- 

- 

Distance 

V 

8 

- 

- 

- 

- 

WFactor 

V 

21 

- 

- 

- 

- 
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TABLE  D.7  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  vn 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

WXPosition 

V 

5 

- 

- 

- 

- 

WYPosition 

V 

5 

- 

- 

- 

- 

Height 

V 

10 

- 

- 

- 

- 

WObserve 

V 

13 

- 

- 

- 

- 

W 

Movement 

V 

12 

- 

- 

- 

- 

Ten-Effect 

V 

15 

- 

- 

- 

- 

Distance 

V 

8 

- 

- 

- 

- 

Restoration 

VI 

97 

7.16 

2/31 

2.0 

0.065 

DeltaFixSup 

VI 

13 

- 

- 

- 

- 

Set 

Restoration 

VI 

23 

7.12 

- 

- 

- 

Change 
Squad  Data 

VI 

61 

- 

- 

- 

- 
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TABLE  D.8  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  vin*^ 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

Others 

0 

258 

- 

0/41 

7.778 

0.190 

Conflict 

0 

93 

- 

- 

- 

- 

UpdateU 

0 

14 

- 

- 

- 

- 

SetKU 

0 

15 

- 

- 

- 

- 

Init 
Variables 

0 

47 

- 

- 

- 

- 

Length  Of 
List 

0 

11 

- 

- 

- 

" 

UpdateVars 

0 

53 

- 

- 

- 

Update 
Nums 

0 

4 

- 

- 

- 

- 

Prepare 
Output 

0 

12 

- 

- 

- 

- 

Ceiling 

o 

9 

- 

- 

- 

- 

Movement 

I 

138 

8.18,  8.35 

4/41 

4.160 

0.101 

SquadPos 

I 

54 

8.2 

- 

- 

- 

Velocity 

I 

25 

- 

- 

- 

- 

Movement 

I 

14 

- 

- 

- 

- 

18 


The  fault  rate  is  41/1366  =0.030  faults  per  line  (including  data 
declarations  in  the  line  count) 
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TABLE  D.8  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  Vffl 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

SetLocation 

I 

45 

8.23 

- 

- 

- 

Observation 

n 

203 

8.15,  8.19,  8.26 

11/41 

5.939 

0.145 

VisContrast 

u 

7 

8.36 

- 

- 

- 

Observation 

II 

60 

8.9 

- 

- 

- 

Height 

n 

6 

8.41 

- 

- 

- 

FindAngle 

II 

37 

- 

- 

- 

- 

CalcAngle 

II 

11 

8.37,  8.38 

- 

- 

- 

FindPoints 

II 

17 

8.39,  8.40 

- 

- 

- 

SumObsJam 

II 

25 

8.14 

- 

- 

- 

Observable 

II 

26 

- 

- 

- 

- 

ResetObser 
ve  Lists 

n 

14 

- 

- 

- 

Attrition 

III 

211 

- 

3/41 

6.361 

0.155 

Attrition 

III 

30 

8.13 

- 

- 

- 

NumOf 
Weapons 

III 

15 

- 

- 

- 

- 

SetAttacked 

in 

19 

- 

- 

- 

- 

UpdateK 

in 

29 

8.48 

- 

- 

- 

CalcCas 

in 

21 

- 

- 

- 

- 

CalcBK 

in 

22 

- 

- 

- 

- 
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TABLE  D.8  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  Vffl 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

UpdateKA 

III 

23 

8.1 

- 

- 

- 

UpdateKU 

III 

21 

- 

- 

- 

- 

ClearAttack 
Lists 

III 

14 

- 

- 

- 

- 

SetStatus 

III 

17 

- 

- 

- 

- 

Communica 
tion 

IV 

358 

8.27,  8.28, 
8.29,  8.30, 
8.31,  8.32, 
8.33,  8.34 

15/41 

10.793 

0.263 

InsertMsg 

IV 

41 

8.5 

- 

- 

- 

Command 

Msg 

IV 

13 

- 

- 

- 

- 

InsertCom 

IV 

17 

- 

- 

- 

- 

ReportMsg 

IV 

7 

- 

- 

- 

- 

InsertRep 

IV 

29 

- 

- 

- 

- 

Communica 
tion 

IV 

7 

- 

- 

- 

- 

Receive 
Delay 

IV 

31 

- 

- 

- 

- 

CalRDelay 

IV 

29 

8.12,  8.50 

- 

- 

- 

QueDelay 

IV 

27 

- 

- 

- 

- 

PutQue 

IV 

39 

8.3,  8.4,  8.6 

- 

- 

- 

ProcessQue 

IV 

28 

- 

- 

- 

- 

ProcessMsg 

IV 

39 

- 

- 

- 

- 

MergeRep 

Msg 

IV 

12 

- 

- 

- 
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TABLE  D.8  FAULT  DISTRIBUTION  TO  FUNCTIONAL  MODULES  IN 

VERSION  vm 


Procedure 

Type 

Lines 

Observed 
Faults 

Relative 
Frequency 

Expected 
Faults 

Relative 
Frequency 

MergeCom 
Msg 

IV 

39 

8.7 

- 

- 

- 

Environment 

V 

110 

- 

7/41 

3.316 

0.081 

Linear 
Distance 

V 

7 

- 

- 

- 

- 

Altitude 

V 

19 

8.42 

- 

- 

- 

WSevFactor 

V 

20 

- 

- 

- 

- 

CalcBI 

V 

24 

8.8.  8.10. 
8.11.8.44 

- 

- 

- 

Terrain  Effect 

V 

21 

- 

- 

- 

- 

Weather 
Move  Effect 

V 

13 

8.45 

- 

- 

- 

WObsEffect 

V 

6 

8.46 

- 

- 

- 

Restoration 

VI 

88 

8.25 

1/41 

2.653 

0.065 

UpdateFS 

VI 

18 

- 

- 

- 

- 

UpdateFF 

VI 

15 

- 

- 

- 

- 

UpdateE 

VI 

55 

- 

- 

- 

- 
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APPENDIX  E:  SHARED  BOUNDING  DIMENSIONS  IN  FUNCTION- 
ALLY CLUSTERED  FAILURE  REGIONS 

In  this  Appendix  we  present  a  detailed  analysis,  whether  functional 
clustering  of  failure  regions  results  in  an  increased  number  of  shared ^^ 
bounding  dimensions. 

The  results,  are  presented  in  the  tables  El  to  E8.  Each  of  the  tables, 
corresponds  to  one  of  the  eight  versions  of  the  CONFLICT  program 
(Shimeall  1991,  1991b).  Each  table  entry,  corresponds  to  a  pair  of  failure 
regions  of  the  same  version  as  in  Appendix  C. 

Table  entries  on  the  main  diagonal,  contain  the  functional  cluster 
identifier  (I,  II,  III,  IV,  V,  VI,  O),  for  the  corresponding  failure  region  (cf 
Chapter  III).  Each  of  the  off  diagonal  entries  contains  the  Jaccard  coefficient 
for  the  identical  dimensions  of  the  two  failure  regions  labeling  the  row  and 
column,  and  the  fault  type  identifier,  in  case  the  regions  belong  to  the  same 
functional  cluster. 


^^       In  this  analysis,  shared  dimensions  are  the  ones  that  correspond  to  the  same 
predicates,  identical. 
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For  example,  entry  (1.1,  1.6)  contains  O/O  which  means  that  the  Jaccard 
Coefficient  for  these  failure  regions  is  equal  to  0,  and  that  both  correspond 
to  faults  of  type  O.  On  the  other  hand,  entry  (1.5,  1.12)  contains  0.154,  so 
the  two  failure  regions  have  a  Jaccard  coefficient  equal  to  0.154,  but 
correspond  to  different  types  of  faults  (VI  and  V  respectively,  in  this  case). 
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TABLE  E.l  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  I 


1.1 

1.2 

1.3 

1.4 

1.5 

1.6 

1.2 

O/O 

0 

- 

- 

- 

- 

1.3 

0 

0 

V 

- 

- 

- 

1.4 

0 

0 

1.333/V 

V 

- 

- 

1.5 

0 

0.100 

0 

0 

VI 

- 

1.6 

O/O 

O/O 

0 

0 

0 

O 

1.7 

O/O 

O/O 

0 

0 

0 

O/O 

1.8 

0 

0.083 

0 

0 

0 

0 

1.9 

0 

0 

0 

0 

0/VI 

O/O 

1.10 

0.333/O 

O/O 

0 

0 

0 

0 

1.11 

0. 333/0 

O/O 

0 

0 

0 

O/O 

1.12 

0 

0.0833 

0/V 

0/V 

0.154 

0 

1.13 

0 

0 

0 

0 

0 

0 

1         1.14 

0 

0 

0 

0 

0 

0 
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TABLE  E.l  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  I 


1.7 

1.8 

1.9 

1.10 

1.11 

1.12 

1.13 

1.7 

0 

- 

- 

- 

- 

- 

- 

1.8 

0 

I 

- 

- 

- 

- 

- 

1.9 

0 

0.063 

VI 

- 

- 

- 

- 

1.10 

o/o 

0 

0 

0 

- 

- 

- 

1.11 

0  /o 

0 

0 

0.580  /O 

0 

- 

- 

1.12 

0 

0 

0 

0 

0 

V 

- 

1.13 

0 

0.182 

0.083 

0 

0 

0 

m 

1.14 

0 

0.067 

0 

0 

0 

0 

0.143/ 

m 

TABLE  E.l  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  I 


I 

1.1 

1.2 

1.3 

1.4 

1.5 

1.6 

1.7 

1.17 

0. 333/0 

O/O 

0 

0 

0 

O/O 

O/O 

1.18 

0 

0.100 

0/V 

0/V 

0.083 

0 

0 

1.19 

0 

0 

0 

0 

0.250 

0 

0 

1.20 

0 

0.056 

0 

0 

0.053 

0 

0 

1.22 

0 

0 

0 

0 

0 

0 

0 

1.23 

O/O 

O/O 

0 

0 

0 

O/O 

O/O 

1.25 

0 

0 

0 

0 

0.053 

0 

0 

1.26 

0 

0 

0 

0 

0 

0 

0 

1.27 

0 

0 

0 

0 

0 

0 

0 

1.28 

0 

0 

0 

0 

0.500/VI 

0 

0 
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TABLE  E.l  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  I 


1.8 

1.9 

1.10 

1.11 

1.12 

1.13 

1.14 

1.17 

0 

0 

0.50/O 

0.50/O 

0 

0 

0 

1.18 

0 

0 

0 

0 

0.25CNV 

0 

0.111 

1.19 

0 

0.154 

0 

0 

0.100 

0.125 

0.067 

1.20 

0.118 

0.048 

0 

0 

0.053 

0.133/III 

0.105 

1.22 

0.056 

0.600 

0 

0 

0 

0.071 

0.048 

1.23 

0 

0 

O/O 

O/O 

0.700 

0 

0 

1.25 

0.050 

0.045 

0 

0 

0 

0.118 

0.109 

1.26 

0 

0 

0 

0 

0 

0 

O/II 

1.27 

0.120 

0.036 

0 

0 

0 

0.143/in 

0.074 
/II 

1.28 

0 

OA^I 

0 

0 

0.143 

0 

0 
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TABLE  E.l  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  I 


1.17 

1.18 

1.19 

1.20 

1.17 

0 

- 

- 

- 

1.18 

0 

V 

- 

- 

1.19 

0 

0.100 

VI 

- 

1.20 

0 

0.100 

0.118 

m 

1.22 

0 

0 

0.067 

0.043 

1.23 

o/o 

0.250 

0.091 

0.050 

1.25 

0 

0 

0.059 

0.421 

1.26 

0 

0 

0 

0 

1.27 

0 

0 

0.045 

0.103 

1.28 

0 

0.071 

0.200 

0.050 

TABLE  E.l  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  I 


1.22 

1.23 

1.25 

1.26 

1.27 

1.28 

1.22 

IV 

- 

- 

- 

- 

- 

1.23 

0 

O 

- 

- 

- 

- 

1.25 

0.042/IV 

0.048 

IV 

- 

- 

- 

1.26 

0 

0 

0 

n 

0 

0 

1.27 

0.033 

0 

0.061 

O/II 

n 

- 

1.28 

0 

0.357 

0.053 

0 

0 

VI 
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TABLE  E.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.1 

2.2 

2.3 

2.4 

2.5 

2.6 

2.1 

IV 

- 

- 

- 

- 

- 

2.2 

0/IV 

IV 

- 

- 

- 

- 

2.3 

0/IV 

0.091/IV 

rv 

- 

- 

- 

2.4 

0/IV 

0.09/IV 

0. 300/1 V 

IV 

- 

- 

2.5 

0/IV 

0.067/IV 

0. 133/1 V 

0.1 33/1 V 

IV 

- 

2.6 

0 

0.091 

0.091 

0.083 

0.063 

V 

2.7 

0.077 

0 

0 

0 

0 

0 

2.8 

0.077 

0 

0 

0 

0 

0 

2.10 

0 

0 

0 

0 

0 

0 

2.11 

0 

0 

0 

0 

0 

0 

1        2.12 

0 

0 

0 

0 

0 

0 

2.13 

0 

0.111 

0.091 

0.091 

0.154 

0 

2.14 

0 

0 

0 

0 

0 

0 

2.15 

0 

0 

0 

0 

0 

0 
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TABLE  E.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.7 

2.8 

2.10 

2.11 

2.12 

2.13 

2.14 

2.15 

2.7 

n 

- 

- 

- 

- 

- 

- 

- 

2.8 

0.636/II 

n 

- 

- 

- 

- 

- 

- 

2.10 

0 

0 

I 

- 

- 

- 

- 

- 

2.11 

0 

0 

O/I 

I 

- 

- 

- 

- 

2.12 

0 

O/II 

0 

0 

n 

- 

- 

- 

2.13 

0 

0 

O/I 

O/I 

0 

I 

- 

- 

2.14 

0 

0 

0 

0 

0 

0 

VI 

- 

2.15 

O/II 

O/II 

0 

0 

O/II 

0.400 

0 

n 

TABLE  E.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.1 

2.2 

2.3 

2.4 

2.5 

2.6 

2.16 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

2.17 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

2.18 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

2.19 

0/IV 

0/IV 

0/IV 

OAV 

0/IV 

0 

2.20 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

2.21 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

2.22 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

2.23 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 
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TABLE  E.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


2.7 

2.8 

2.10 

2.11 

2.12 

2.13 

2.14 

2.15 

2.16 

0 

0 

0 

0 

0 

0 

0.400 

0.400 

2.17 

0 

0 

0 

0 

0 

0 

0.400 

0.400 

2.18 

0 

0 

0 

0 

0 

0 

0.400 

0.400 

2.19 

0 

0 

0 

0 

0 

0 

0 

0 

2.20 

0 

0 

0 

0 

0 

0 

0 

0 

2.21 

0 

0 

0 

0 

0 

0 

0 

0 

2.22 

0 

0 

0 

0 

0 

0 

0 

0 

2.23 

0 

0 

0 

0 

0 

0 

0 

0 

2.25 

0 

0 

0 

0 

0 

0 

0 

0 

2.26 

O/II 

O/II 

0 

0 

0 

0 

0 

O/II 

2.30 

0 

0 

0 

0 

0 

0 

0 

0 

2.31 

0 

0 

0 

0 

0 

0 

0 

0 

TABLE  E.2  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  n 


==^^=== 

2.1 

2.2 

2.3 

2.4 

2.5 

2.i 

2JS 

(Viv 

(WV 

(vrv 

(VIV 

CffV 

0 

1.U 

0 

0 

0 

0 

0 

0 

2.30 

0 

O.Ul 

0.111 

0.100 

0.071 

0.333A' 

2.31 

0 

0 

0 

0 

0 

0/V 
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TABLE  E.2 


JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 
VERSION  n 


2.16 

2.17 

2.18 

2.19 

2.20 

2.16 

IV 

- 

- 

- 

- 

2.17 

0.40 

IV 

- 

- 

- 

2.18 

0/IV 

0.400/IV 

IV 

- 

- 

2.19 

0/IV 

0/IV 

0/IV 

IV 

- 

2.20 

0/IV 

0/IV 

0/IV 

0.571/IV 

IV 

2.21 

0/IV 

0/IV 

0/IV 

0.571/IV 

0.571/IV 

2.22 

0/IV 

0/IV 

OAV 

0.571AV 

0.571/IV 

2.23 

0/IV 

0/IV 

0/IV 

0.571/IV 

0.571/IV 

2.25 

0/IV 

0/IV 

O/IV 

0/IV 

0/IV 

2.26 

0 

0 

0 

0 

0 

2.30 

0 

0 

0 

0 

0 

2.31 

0 

0 

0 

0 

0 

TABLE  E.2 


JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 
VERSION  n 


2.21 

2.22 

2.23 

2.24 

2.30 

2.21 

IV 

- 

- 

- 

- 

2.22 

0.571  /IV 

IV 

- 

- 

- 

2.23 

0.571  /IV 

0.571  /IV 

IV 

- 

- 

2.25 

0/IV 

0/IV 

0/IV 

IV 

- 

2.26 

0 

0 

0 

0 

n 

2.30 

0 

0 

0 

0 

0 

V 

2.31 

0 

0 

0 

0 

0 

0/V 

V 
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TABLE  E.3  :   JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  m 


3.1 

3.2 

3.3 

3.4 

3.5 

3.6 

3.7 

3.8 

3.9 

3.10 

3.1 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

3.2 

0.60/lV 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

3.3 

0.25 
/IV 

0.286 
/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

3.4 

0/IV 

0/IV 

0/IV 

IV 

- 

- 

- 

- 

- 

- 

3.5 

0 

0 

0 

0 

0 

- 

- 

- 

- 

- 

3.6 

0.143 

0.200 

0 

0 

0 

I 

- 

- 

- 

- 

3.7 

0.067 

0.071 

0 

0 

0 

0.07 

m 

- 

- 

3.8 

0 

0 

0 

0 

0 

0 

O/III 

m 

- 

- 

3.9 

0 

0 

0 

0 

0 

0 

0.48/ni 

O/III 

m 

- 

3.10 

0 

0 

0 

0 

0- 
.17/0 

0 

0 

0 

0 

0 

3.11 

0.111 

0.125 

0.091 

0 

0 

0.13 

0.059 

0 

0 

0 

3.12 

0.111 

0.125 

0 

0 

0 

0.11 

0.188 

0 

0.087 

0    1 

3.13 

0 

0 

0 

0 

0 

0 

0 

0 

0.042 

0 

3.15 

0.077 
/IV 

0.083 

/IV 

0.364 
/IV 

0.100 
/IV 

O/O 

0 

0 

0 

0.040 

O/O 

3.16 

0.11 

0.125 

0.091 

0 

0 

0.13 

0.059 

0 

0 

0 

3.17 

0 

0 

0 

0 

O/O 

0 

0 

0 

0 

O/O 

3.18 

0.067 

0.077 

0 

0 

0 

0.09 

0.714 
All 

O/III 

0.429 

/m 

0 

3.19 

0 

0 

0 

0 

0 

0 

0/in 

O/III 

O/III 

0 

3.21 

0 

0 

0 

0 

O/O 

0 

0 

0 

0 

O/O 

3.22 

0.125 

0.167 

0 

0 

0 

0.20 

0.063 

0 

0.048 

0 

3.23 

0.182 
/IV 

0.200 
/IV 

0.077 
/IV 

0/IV 

0.- 
0833 

0.091 

0.111 

0 

0.040 

0.111 
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TABLE  E.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  ID 


3.11 

3.12 

3.13 

3.15 

3.16 

3.17 

3.18 

3.19 

3.2 

1 

3.2 

2 

3.2 
3 

3.11 

V 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

3.12 

0.111 

/v 

V 

- 

- 

- 

- 

- 

- 

- 

- 

- 

3.13 

0 

0 

n 

- 

- 

- 

- 

- 

- 

- 

- 

3.15 

0.067 

0 

0 

O/IV 

- 

- 

- 

- 

- 

- 

3.16 

0.333/V 

0.111 

/v 

0 

0.06 

7 

V 

- 

- 

- 

- 

- 

3.17 

0 

0 

0 

O/O 

0 

0 

- 

- 

- 

- 

- 

3.18 

0.063 

0.059 

0 

0.05 
6 

0.06 

3 

0 

m 

- 

- 

- 

- 

3.19 

0 

0 

0 

0 

0 

0 

O/III 

m 

3.21 

0 

0 

0 

O/O 

0 

O/O 

0 

0 

0 

- 

- 

3.22 

0.111 

0.100 

0 

0 

0.11 

1 

0 

0.15 
4 

0 

0 

VI 

- 

3.23 

0.077 

0.167 

0 

0.06- 
3/IV 

0 

0 

0.05 
3 

0 

0 

0.8 

33 

IV 
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TABLE  E.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  m 


3.1 

3.2 

3.3 

3.4 

3.5 

3.6 

3.7 

3.8 

3.9 

3.10 

3.24 

0.143 

0.167 

0.167 

0 

0.286- 

/o 

0 

0.071 

0 

0 

0.2- 
5/0 

3.25 

0.125 

0.167 

0.250 

0 

0 

0.250 
/I 

0.067 

0 

0 

0 

3.26 

0 

0 

0.125 

0.125 

0.571 

/o 

0 

0 

0 

0 

0.16- 
7/0 

3.28 

0 

0.125 

0 

0.100 

0.222 

0.125 

0.063 

0 

0 

0 

3.32 

0 

0 

0.167 

0 

o/o 

0 

0 

0 

0 

o/o 

3.33 

0/IV 

0/IV 

0.143 
/IV 

0/IV 

0 

0 

0 

0 

0 

0 

3.34 

0 

0 

0.143 

0 

0 

0 

0 

0 

0 

0 

3.35 

0/IV 

0/IV 

0.143 
/IV 

0/IV 

0 

0 

0 

0 

0 

0 

3.36 

0/IV 

0/IV 

0.167 
/IV 

0/IV 

0 

0 

0 

0 

0 

0 

3.37 

0/IV 

0/IV 

0.143 

/IV 

0/IV 

0 

0 

0 

0 

0 

0 

3.38 

0.111 

/IV 

0/IV 

0/IV 

0.125 

0.222 

0 

0.067 

0 

0 

0.167 

3.39 

0/IV 

0/IV 

0/IV 

0.125 

0.222 

0 

0 

0 

0 

0.200 

3.40 

0/IV 

0/IV 

0/IV 

0.125 

0.222 

0 

0 

0 

0 

0.200 
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TABLE  E.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  in 


3.11 

3.12 

3.13 

3.15 

3.16 

3.17 

3.18 

3.19 

3.21 

3.22 

3.2 
3 

3.24 

0 

0 

0 

0.10 

/o 

0 

O/O 

0.07 
7 

0 

O/O 

0 

0.1 
82 

3.25 

0.11 

1 

0.10 
0 

0 

0.09 

1 

0.11 
1 

0 

0.08 
3 

0 

0 

0.16 

7 

0.0 
83 

3.26 

0 

0 

0 

0.09 
1/0 

0 

O/O 

0 

0 

O/O 

0 

0.0 
83 

3.28 

0.11 

1/V 

0.11- 
1/V 

0 

0 

0.11 

1/V 

0 

0.06 
3 

0 

0 

0.11 
1 

0.0 
83 

3.32 

0 

0 

0 

0.10 
O/O 

0 

O/O 

0 

0 

0.33 
3/0 

0 

0 

3.33 

0 

0 

0 

0.09 

1 

0 

0 

0 

0 

0.25 
0 

0 

/VI 

0 

3.34 

0 

0 

O/II 

0.09 
1 

0 

0 

0 

0 

0.25 
0 

0 

0 

3.35 

0 

0 

0 

0.09 
1 

0 

0 

0 

0 

0.25 

0 

0 

TABLE  E.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  m 


3.1 

3.2 

3.3 

3.4 

3.5 

3.6 

3.7 

3.8 

3.9 

3.10 

3.41 

0.071 
/IV 

0/IV 

0/IV 

0.125 

0.222 

0 

0 

0 

0 

0.200 

3.42 

0/IV 

0.111 
/IV 

0/IV 

0.126 

0.222 

0 

0 

0 

0 

0.200 

3.43 

0.091 

0 

0 

0.167 

0 

0.143 

0.118 

0 

0.042 

0 

3.44 

0 

0.083 

0 

0 

0 

0.091 

0.770 

/in 

0/m 

0.381 
/III 

0 

3.45 

0 

0 

0 

0 

0.056 

0 

O/III 

O/III 

0.529 

m 

0.381 
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TABLE  E.3  :   JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  in 


3.11 

3.12 

3.13 

3.1 

5 

3.1 
6 

3.1 

7 

3.19 

3.18 

3.21 

3.22 

3.23 

3.36 

0 

0 

0 

0.091 

/IV 

0 

0 

0 

0 

0.250 

0 

0/IV 

3.37 

0 

0 

0 

0.091 

/TV 

0 

0 

0 

0 

0.250 

0 

0/IV 

3.38 

0 

0 

0 

0 
/IV 

0 

0 

0 

0.063 

0 

0 

0.08- 
3/IV 

3.39 

0 

0 

0 

0 
/IV 

0 

0 

0 

0 

0 

0 

0.08- 
3/IV 

3.40 

0 

0 

0 

0 

/IV 

0 

0 

0 

0 

0 

0 

0.08- 
3/IV 

3.41 

0 

0 

0 

0 
/IV 

0 

0 

0 

0 

0 

0 

0.08- 
3/IV 

3.42 

0 

0 

0 

0 
/IV 

0 

0 

0 

0 

0 

0 

0.08- 
3/IV 

3.43 

0.083 

0.077 

O/II 

0 

0.08 
3 

0 

0 

0.143 

0 

0.111 

0.067 

3.44 

0.067 

0.063 

0 

0 

0.06 

7 

0 

O/III 

0.910 

/III 

0 

0.077 

0.056 

3.45 

0 

0.05 

0 

0 

0 

0 

O/III 

0.471 

m 

0 

0 

0 
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TABLE  E.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  in 


3.24 

3.25 

3.26 

3.28 

3.32 

3.33 

3.34 

3.35 

3.24 

0 

- 

- 

- 

- 

- 

- 

- 

3.25 

0 

I 

- 

- 

- 

- 

- 

- 

3.26 

0.333 

/o 

0 

0 

- 

- 

- 

- 

- 

3.28 

0.125 

0.111 

0.333 

V 

- 

- 

- 

- 

3.32 

O/O 

0 

0.167  /O 

0 

0 

- 

- 

- 

3.33 

0 

0 

0.143 

0 

0.500 

VI 

- 

- 

3.34 

0 

0 

0.143 

0 

0.500 

0.400 

n 

- 

3.35 

0 

0 

0.143 

0 

0.500 

0.400 

0.400 

IV 

3.36 

0 

0 

0.143 

0 

0.500 

0.400 

0.400 

0.40/rv 

3.37 

0 

0 

0.143 

0 

0.500 

0.400 

0.400 

0.40/IV 

3.38 

0.286 

0 

0.375 

0.200 

0 

0 

0 

0/IV 

3.39 

0.143 

0 

0.250 

0.100 

0 

0 

0 

0/IV 

3.40 

0.143 

0 

0.250 

0.100 

0 

0 

0 

0/IV 

3.41 

0.143 

0 

0.250 

0.100 

0 

0 

0 

0/IV 

3.42 

0.143 

0 

0.250 

0.100 

0 

0 

0 

0/IV 

3.43 

0 

0.125 

0 

0.083 

0 

0 

O/II 

0 

3.44 

0.077 

0.083 

0 

0.063 

0 

0 

0 

0 

3.45 

0.059 

0 

0.059 

0.053 

0 

0 

0 

0 
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TABLE  E.3  :    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  m 


3.36 

3.37 

3.38 

3.39 

3.40 

3.41 

3.42 

3.43 

3.44 

3.45 

3.36 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

3.37 

0.400 
/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

3.38 

0/IV 

0 

/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

3.39 

0/IV 

0 

/IV 

0.28- 
6/IV 

IV 

- 

- 

- 

- 

- 

- 

3.40 

0/IV 

0 
/IV 

0.286 
/IV 

0.500 
/IV 

IV 

- 

- 

- 

- 

- 

3.41 

0/IV 

0 

/IV 

0.286 

/IV 

0.50- 
0/IV 

0.500 
/IV 

IV 

- 

- 

- 

- 

3.42 

0/IV 

0 
/IV 

0.286 

/IV 

0.500 
/IV 

0.500 
/IV 

0.500 
/IV 

IV 

- 

- 

- 

3.43 

0 

0 

0 

0.091 

0.091 

0.091 

0.091 

n 

- 

- 

3.44 

0 

0 

0.067 

0 

0 

0 

0 

0.143 

m 

- 

3.45 

0 

0 

0.105 

0.053 

0.053 

0.053 

0.053 

0.053 

O/III 

m 
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TABLE  E.4  :    JACCARD  COEFFICIENTS  OF    FAILURE  REGIONS  OF 


VERSION  IV 


4.1 

4.2 

4.3 

4.4 

4.5 

4.6 

4.7 

4.8 

4.9 

4.10 

4.11 

4.1 

2 

4.1 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.2 

0 

V 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.3 

0 

0.118 

m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.4 

0 

0 

0.045 

n 

- 

- 

- 

- 

- 

- 

- 

- 

4.5 

0 

0 

0.063 

0.071 
/II 

n 

- 

- 

- 

- 

- 

- 

- 

4.6 

0 

0 

0 

O/II 

O/II 

n 

- 

- 

- 

- 

- 

- 

4.7 

0 

0 

0.059 

0.071 
/II 

0.16- 
7/II 

O/II 

n 

- 

- 

- 

- 

- 

4.8 

0 

0 

0.167 
All 

0.059 

0.125 

0 

0.111 

/II 

m 

- 

- 

- 

- 

4.9 

0 

0 

0.063 

0.07- 
l/II 

0.25- 
O/II 

o/n 

0.167 
AI 

0.125 

n 

- 

- 

- 

4.10 

0 

0 

0.056 

0.06- 
3/II 

O/II 

O/II 

0.125 
/II 

0.091 

0.12- 
5/II 

n 

- 

- 

4.11 

o/rv 

0.053 

0.043 

0 

0 

0 

0 

0 

0 

0 

IV 

- 

4.12 

0 

0 

0.063 

0.21- 
4/II 

0.16- 

7/n 

O/II 

0.12- 
5/II 

0.100 

0.167 
/II 

0.10- 
0/U 

0 

n 

4.13 

O/IV 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4.14 

O/IV 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4.15 

0.500 

0 

0.091 

0 

0 

0 

0 

0.214 

0 

0 

0 

0 

4.16 

0 

0.071 

0 

0.18- 
8/II 

o/u 

O/II 

OAI 

0 

O/II 

OAI 

0 

0.0- 

8/n 

4.18 

0 

0.100 

0 

O/II 

O/II 

O/II 

OAI 

0 

O/II 

O/II 

0 

o/n 

4.19 

0 

0.100 

O/III 

O/II 

0/n 

O/II 

OAI 

O/III 

O/II 

O/II 

0 

O/II 

4.20 

0 

0 

0 

OAI 

o/n 

O/II 

O/II 

0 

O/II 

O/II 

0 

o/n 
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TABLE  E.4  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  IV 


4.13 

4.14 

4.15 

4.16 

4.18 

4.19 

4.20 

4.2 

1 

4.2 

2 

4.24 

4.28 

4.13 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.14 

0.800 
/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

4.15 

0/IV 

0/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

4.16 

0 

0 

0 

n 

- 

- 

- 

- 

- 

- 

- 

4.18 

0 

0 

0 

0.09- 
l/II 

n 

- 

- 

- 

- 

- 

" 

4.19 

0 

0 

0 

0.07- 
l/II 

0.12- 
5/II 

n/ffl 

- 

- 

- 

- 

4.20 

0 

0 

0 

O/II 

O/II 

O/II 

n 

4.21 

0 

0 

0 

0.063 

0.083 

0.077 

0 

VI 

- 

- 

- 

4.22 

0 

0 

0 

O/II 

O/II 

O/II 

O/II 

0 

n 

- 

- 

4.24 

0/IV 

0/IV 

0/IV 

0.083 

0.167 

0.143 

0 

0 

0 

IV 

- 

4.26 

0/IV 

0/IV 

0/IV 

0.071 

0.125 

0.111 

0 

0.077 

0 

0.60 
/IV 

IV 

TABLE  E.4  :    JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  IV 


4.1 

4.2 

4.3 

4.4 

4.5 

4.6 

4.7 

4.8 

4.9 

4.10 

4.11 

4.1 

2 

4.21 

0 

0.071 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

4.22 

0 

0 

0 

O/II 

O/II 

O/II 

O/II 

0 

0/n 

O/II 

0 

O/II 

4.24 

0/IV 

0.100 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

0 

4.26 

0/IV 

0.083 

0 

0 

0 

0 

0 

0 

0 

0 

O/.IV 

0 
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TABLE  E.5:    JACCARD     COEFFICIENTS  OF  FAILURE  REGIONS  OF 


VERSION  V 


5.1 

5.2 

5.3 

5.4 

5.5 

5.6 

5.7 

5.8 

5.9 

5.1 

n 

- 

- 

- 

- 

- 

- 

- 

- 

5.2 

0 

IV 

- 

- 

- 

- 

- 

- 

- 

5.3 

0 

0 

O 

- 

- 

- 

- 

- 

5.4 

0 

0.053/rV 

0 

IV 

- 

- 

- 

- 

- 

5.5 

0 

0.059/IV 

0 

0.375/IV 

IV 

- 

- 

- 

- 

5.6 

0 

0 

0.167  /O 

0 

0 

0 

- 

- 

- 

5.7 

0 

0 

O/O 

0 

0 

O/O 

0 

- 

- 

5.8 

0 

0.111/IV 

0 

0.059/IV 

0.067/IV 

0 

0 

IV 

- 

5.9 

0 

0.05: 

O/O 

0 

0 

O/O 

O/O 

0 

0 

5.10 

0 

0 

O/O 

0 

0 

O/O 

O/O 

0 

0.143 

/o 

5.11 

0 

0 

O/O 

0 

0 

O/O 

O/O 

0 

O/O 

5.12 

0 

0 

O/O 

■J 

0 

O/O 

O/O 

0 

O/O 

5.13 

0 

0.050 

0 

0.083 

0.100 

0 

0 

0.056 

0 

5.14 

0 

0 

O/O 

0 

0 

O/O 

O/O 

0 

O/O 

5.15 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.16 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.17 

O/II 

0 

0 

0 

0 

0 

0 

0 

0 

5.18 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.19 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

0 
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TABLE  E.5:  JACCARD     COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  V 


5.10 

5.11 

5.12 

5.13 

5.14 

5.15 

5.16 

5.17 

5.18 

5.1 

7 

5.10 

0 

- 

- 

- 

- 

- 

- 

- 

- 

- 

5.11 

o/o 

0 

- 

- 

- 

- 

- 

- 

- 

- 

5.12 

o/o 

0.20- 
O/O 

0 

- 

- 

- 

- 

- 

- 

- 

5.13 

0 

0 

0 

V 

- 

- 

- 

- 

- 

- 

5.14 

o/o 

o/o 

o/o 

0.8 

O 

- 

- 

- 

- 

5.15 

0 

0 

0 

0 

0 

m 

- 

- 

- 

- 

5.16 

0 

0 

0 

0 

0 

0.71- 

4/in 

m 

- 

- 

- 

5.17 

0 

0 

0 

0 

0 

0 

0 

n 

- 

- 

5.18 

0 

0 

0 

0 

0 

0 

0 

0 

VI 

- 

5.19 

0 

0 

0 

0 

0 

0 

0 

0 

0 

IV 
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TABLE  E.5:  JACCARD     COEFFICIENTS  OF    FAILURE  REGIONS  OF 

VERSION  V 


5.1 

5.2 

5.3 

5.4 

5.5 

5.6 

5.7 

5.8 

5.20 

O/II 

0 

0 

0 

0 

0 

0 

0 

5.21 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.22 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.23 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.24 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.25 

0 

OAV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.26 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.27 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.28 

0 

0 

o/o 

0 

0 

o/o 

o/o 

0 

5.30 

0 

0 

o/o 

0 

0 

o/o 

o/o 

0 

5.31 

0 

0 

o/o 

0 

0 

o/o 

o/o 

0 

5.32 

0 

0 

o/o 

0 

0 

o/o 

o/o 

0 

5.33 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.34 

O/II 

0/IV 

0 

0/IV 

0/IV 

0 

0 

OAV 

5.35 

0 

0 

0 

0 

0 

0 

0 

0 

5.37 

O/II 

0 

0 

0 

0 

0 

0 

0 

5.39 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.40 

0 

0/IV 

0 

0/IV 

0/IV 

0 

0 

0/IV 

5.41 

O/II 

0 

0 

0 

0 

0 

0 

0 

5.42 

0 

0 

o/o 

0 

0 

o/o 

o/o 

0 

5.43 

O/II 

0 

0 

0 

0 

0 

0 

0 
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TABLE  E.5:  JACCARD     COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  V 


5.9 

5.10 

5.11 

5.12 

5.13 

5.14 

5.1 
5 

5.1 
6 

5.1 

7 

5.18 

5.19 

5.20 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.21 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.22 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.23 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.24 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.25 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.26 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.27 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.28 

o/o 

o/o 

o/o 

o/o 

0 

o/o 

0 

0 

0 

0 

0 

5.30 

o/o 

o/o 

o/o 

o/o 

0 

o/o 

0 

0 

0 

0 

0 

5.31 

o/o 

o/o 

o/o 

o/o 

0 

o/o 

0.130 

0.26 

1 

0 

0 

0 

5.32 

o/o 

o/o 

o/o 

o/o 

0 

o/o 

0 

0 

0 

0 

0 

5.33 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.34 

0 

0 

0 

0 

0 

0 

0 

0 

O/II 

0 

0/IV 

5.35 

0 

0 

0 

0 

0 

0 

0.040 

0.04 
0 

0 

0/VI 

0 

5.37 

0 

0 

0 

0 

0 

0 

0 

0 

O/II 

0 

0 

5.39 

0 

0 

0 

0 

0 

0 

0.083 

0.08 
3 

0 

0 

0/IV 

5.40 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0/IV 

5.41 

0 

0 

0 

0 

0/V 

0 

0 

0 

O/II 

0 

0 

5.42 

o/o 

o/o 

o/o 

o/o 

0 

o/o 

0 

0 

0 

0 

0 

5.43 

0 

0 

0 

0 

0 

0 

0 

0 

O/II 

0 

0 
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TABLE  E.5:    JACCARD 


COEFFICIENTS  OF  FAILURE  REGIONS  OF 
VERSION  V 


5.19 

5.20 

5.21 

5.22 

5.23 

5.24 

5.25 

5.26 

5.27 

5.20 

0 

n 

- 

- 

- 

- 

- 

- 

- 

5.21 

0.444/IV 

0 

IV 

- 

- 

- 

- 

- 

- 

5.22 

0/IV 

0.286 

0/IV 

IV 

- 

- 

- 

- 

- 

5.23 

0.444/IV 

0 

0.444/IV 

0/IV 

IV 

- 

- 

- 

- 

5.24 

0/IV 

0.286 

0/IV 

0.286/IV 

0/IV 

IV 

- 

- 

- 

5.25 

0.444/IV 

0 

0.444/IV 

0/IV 

0.444/IV 

0/IV 

IV 

- 

- 

5.26 

0/IV 

0.286 

0/IV 

0.286/IV 

0/IV 

0.286/1 
V 

0/IV 

IV 

- 

5.27 

0.444/IV 

0 

0.444/IV 

0/IV 

0.444/IV 

0/IV 

0.444/1 
V 

0/IV 

IV 

5.28 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.30 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.31 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.32 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.33 

0/IV 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

5.34 

0/IV 

O/II 

0/IV 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

5.35 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.37 

0 

O/II 

0 

0 

0 

0 

0 

0 

0 

5.39 

0/IV 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

5.40 

0/IV 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

5.41 

0 

O/II 

0 

0 

0 

0 

0 

0 

0 

5.42 

0 

0 

0 

0 

0 

0 

0 

0 

0 

5.43 

0 

O/II 

0 

0 

0 

0 

0 

0 

0 
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TABLE  E.5:   JACCARD     COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  V 


5.28 

5.30 

5.31 

5.32 

5.33 

5.34 

5.3 
5 

5.3 

7 

5.3 
9 

5.4 
0 

5.4 

1 

S.4 

2 

5.28 

0 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

5.30 

O/O 

0 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

5.31 

o/o 

o/o 

0 

- 

- 

- 

- 

- 

- 

- 

- 

- 

5.32 

O/O 

o/o 

o/o 

0 

- 

- 

- 

- 

- 

- 

- 

- 

5.33 

0 

0 

0 

0 

IV 

- 

- 

- 

- 

- 

- 

- 

5.34 

0 

0 

0 

0 

0/IV 

IV 

/n 

- 

- 

- 

- 

- 

- 

5.35 

0 

0 

0.053 

0 

0 

0.214 

VI 

- 

- 

- 

- 

5.37 

0 

0 

0 

0 

0 

O/II 

0 

n 

- 

- 

- 

- 

5.39 

0 

0 

0.125 

0 

0 
/TV 

0 
/TV 

0.063 

0 

IV 

- 

- 

- 

5.40 

0 

0 

0 

0 

0/IV 

O/I 

V 

0 

0 

0.05- 
3/IV 

IV 

- 

- 

5.41 

0 

0 

0 

0 

0 

O/II 

0 

O/II 

0 

0 

n/v 

- 

5.42 

o/o 

o/o 

o/o 

o/o 

0 

0 

0 

0 

0 

0 

0 

0 

5.43 

0 

0 

0 

0 

0 

O/II 

0 

O/II 

0 

0 

O/II 

0 
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TABLE  E.6  :    JACCARD    COEFFICIENTS  OF  FAILURE  REGIONS  OF 


VERSION  VI 


6.1 

6.2 

6.3 

6.4 

6.5 

6.6 

6.7 

6.8 

6.1 

n 

- 

- 

- 

- 

- 

- 

- 

1      ^'^ 

0 

0/IV 

- 

- 

- 

- 

- 

- 

6.3 

0.357 

/II 

0 

n 

- 

- 

- 

- 

- 

6.4 

0 

o/o 

0 

0 

- 

-' 

- 

- 

6.5 

0 

0 

0 

0 

m 

- 

- 

- 

6.6 

0.077 

0 

0.063 

0 

O/III 

m 

- 

- 

6.7 

0.056 

o/o 

0.048 

o/o 

0 

0.091 

O 

- 

6.8 

0.167 

0.143/O 

0 

o/o 

0 

0.091 

0.50/O 

0 

6.9 

0.059 

0 

0.05 

0 

0/in 

O.l/III 

0.286 

0.286 

6.11 

0 

0.143/IV 

0 

0 

0 

0 

0 

0 

6.12 

0 

0/IV 

0 

0 

0 

0 

0 

0 

6.13 

0 

0/IV 

0 

0 

0 

0 

0 

0 

6.14 

0 

0/IV 

0 

0 

0 

0 

0 

0 

6.15 

0 

o.iii/rv 

0 

0 

0 

0 

0 

0 

6.16 

0.364AI 

0 

0.286/II 

0 

0 

0.125 

0.083 

0.250 

6.17 

0.056 

0 

0.048 

0 

o/in 

0.083/ni 

0.091 

0.091 

6.18 

0 

0/IV 

0 

0 

0 

0 

0 

0 

6.20 

0 

O/O 

0 

o/o 

0 

0 

O/O 

O/O 

6.23 

0.056 

0 

0.048 

0 

O/III 

0.091AII 

0.091 

0.091 

6.24 

0 

0/IV 

0 

0 

0 

0 

0 

0 
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TABLE  E.6  :    JACCARD    COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  VI 


6.9 

6.11 

6.12 

6.13 

6.14 

6.1 

5 

6.1 
6 

6.1 

7 

6.18 

6.20 

6.2 

3 

6.2 
4 

6.9 

m 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

6.11 

0 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

6.12 

0 

0/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

6.13 

0 

0/IV 

0/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

6.14 

0 

0/IV 

0.059 

0/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

6.15 

0 

0.143 

/iv 

0.071 
AV 

0/IV 

O/I 
V 

IV 

- 

- 

- 

- 

- 

- 

6.16 

0.071 

0 

0 

0 

0 

0 

n 

- 

- 

- 

- 

- 

6.17 

0.067 

/m 

0 

0 

0 

0 

0 

0.07 
7 

m 

- 

- 

- 

- 

6.18 

0 

0/IV 

0.067 
/IV 

0/IV 

0.40- 
O/IV 

0 

0 

0 

IV 

- 

- 

- 

6.20 

0 

0 

0 

0 

0 

0 

0 

0 

0 

o 

- 

- 

6.23 

0.071 
/III 

0 

0 

0 

0 

0 

0.07 
7 

0.73 
/III 

0 

0 

m 

- 

6.24 

0 

0.071 
/IV 

0.118 
/IV 

0/IV 

0.27- 
8/IV 

0.063 
/IV 

0 

0 

0.46 

7/IV 

0 

0 

" 
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TABLE  E.7:    JACCARD     COEFFICIENTS  OF    FAILURE  REGIONS 

OF  VERSION  Vn 


1 


7.1 

7.2 

7.3 

7.4 

7.5 

7.6 

7.7 

7.1 

n 

- 

- 

- 

- 

- 

- 

7.2 

0.083 

m 

- 

- 

- 

- 

- 

7.3 

0.083 

0.8 18/1 II 

m 

- 

- 

- 

- 

7.4 

0 

0 

0 

IV 

- 

- 

- 

7.5 

0. 167/11 

0 

0 

0.071 

n 

- 

- 

7.6 

0 

0 

0 

0 

0 

0 

- 

7.7 

0 

0 

0 

0 

0 

0.333/O 

o 

7.8 

0 

0 

0 

0 

0 

0.333/O 

0.500/O 

7.9 

0 

0 

0 

0. 333/1 V 

0 

0 

0 

7.10 

O/II 

0.231 

0.214 

0 

O/II 

0 

0 

7.11 

0 

0 

0 

0 

0 

0.200 

0.167 

7.12 

0 

0 

0 

0 

0 

0 

0 

7.13 

O/II 

0 

0 

0 

O/II 

0 

0 

7.14 

0 

0 

0 

0.143 

0 

O/O 

O/O 

7.15 

0.200 

0.091/1  II 

0.083/III 

0 

0 

0 

0 

7.16 

0 

0 

0 

0 

0 

0 

0       1 
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TABLE  E.7:    JACCARD     COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.8 

7.9 

7.10 

7.11 

7.12 

7.13 

7.14 

7.15 

7.16 

7.8 

O 

- 

- 

- 

- 

- 

- 

- 

- 

7.9 

0 

IV 

- 

- 

- 

- 

- 

- 

- 

7.10 

0 

0 

n 

- 

- 

- 

- 

- 

- 

7.11 

0.167 

0 

0 

V 

- 

- 

- 

- 

- 

7.12 

0 

0 

0 

0 

VI 

- 

- 

- 

- 

7.13 

0 

0 

O/II 

0 

0 

n 

- 

- 

- 

7.14 

O/O 

0 

0 

0 

0.100 

0 

0 

- 

- 

7.15 

0 

0 

0 

0 

0 

0 

0 

m 

- 

I     7.16 

0 

0 

0 

0 

OA^I 

0 

0 

0 

VI 
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TABLE  E.7:    JACCARD     COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.1 

7.2 

7.3 

7.4 

7.5 

7.6 

7.17 

O/II 

0 

0 

0 

O/II 

0 

7.18 

0 

0 

0 

0/IV 

0 

0 

7.19 

0 

0 

0 

0/IV 

0 

0 

7.20 

0 

0 

0 

0/IV 

0 

0 

1        "^-^l 

0 

0 

0 

0/IV 

0 

0 

7.22 

0 

0 

0 

0/IV 

0 

0 

7.23 

0 

0 

0 

0/IV 

0 

0 

7.24 

0 

0 

0 

0/IV 

0 

0 

7.25 

0 

0 

0 

0/IV 

0 

0 

7.27 

0.100 

0.200/1  II 

0.273/III 

0 

0 

0 

7.28 

0 

0 

0 

0/IV 

0 

o/o 

7.29 

0.077 

0.455/1  II 

0.417/in 

0 

0 

0 

7.32 

0.048/II 

0.038 

0.037 

0 

O/II 

0 

7.33 

0 

0 

0 

0 

0 

0.143 

7.35 

0.077/II 

0.333 

0.400 

0 

O/II 

0 
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TABLE  E.7:  JACCARD     COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.7 

7.8 

7.9 

7.10 

7.11 

7.12 

7.13 

7.1 
4 

7.15 

7.16 

7.17 

0 

0 

0 

O/II 

0 

0 

O/II 

0 

0 

0.400 

7.18 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0.400 

7.19 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0.400 

7.20 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0.400 

7.21 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0 

7.22 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0 

7.23 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0 

7.24 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0 

7.25 

0 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0 

7.27 

0 

0 

0 

0.077 

0 

0 

0 

0 

0.111/ni 

0 

7.28 

O/O 

O/O 

0/IV 

0 

0 

0 

0 

O/O 

0 

0 

7.29 

0 

0 

0 

0.231 

0 

0 

0 

0 

0.083 

0 

7.32 

0 

0 

0 

O/II 

0 

0 

O/II 

0 

0.050 

0 

7.33 

0.125 

0.125 

0 

0 

0.333 

0 

0 

0 

0 

0 

7.35 

0 

0 

0 

0.20- 
O/II 

0 

0 

O/II 

0 

0.091 

0 
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TABLE  E.7:  JACCARD     COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.17 

7.18 

7.19 

7.20 

7.21 

7.17 

n 

- 

- 

- 

- 

7.18 

0.400 

IV 

- 

- 

- 

7.19 

0.400 

0.400/IV 

IV 

- 

- 

7.20 

0.400 

0.400/IV 

0.400/IV 

IV 

- 

7.21 

0 

OAV 

0/IV 

0/IV 

IV 

7.22 

0 

0/IV 

0/IV 

0/IV 

0.571/IV 

7.23 

0 

0/IV 

0/IV 

0/IV 

0.571/IV 

7.24 

0 

0/IV 

0/IV 

0/IV 

0.571AV 

7.25 

0 

0/IV 

0/IV 

0/IV 

0.429/IV 

7.27 

0 

0 

0 

0 

0 

7.28 

0 

0/IV 

0/IV 

0/IV 

0/IV 

7.29 

0 

0 

0 

0 

0 

7.32 

O/II 

0 

0 

0 

0 

7.33 

0 

0 

0 

0 

0 

7.35 

O/II 

0 

0 

0 

0 

166 


TABLE  E.7:  JACCARD      COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vn 


7.22 

7.23 

7.24 

7.25 

7.27 

7.28 

7.29 

7.32 

7.3 
3 

7.35 

7.22 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

7.23 

0.571 

/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

7.24 

0.571 
/IV 

0.571 
/IV 

IV 

- 

- 

- 

- 

- 

- 

- 

7.25 

0.429 

/IV 

0.429 
/IV 

0.429 
/IV 

IV 

- 

- 

- 

- 

- 

- 

7.27 

0 

0 

0 

0 

m 

- 

- 

- 

- 

- 

7.28 

0/IV 

0/IV 

0/IV 

0/IV 

0 

O/IV 

- 

- 

- 

- 

7.29 

0 

0 

0 

0 

0.182 
/III 

0 

m 

- 

- 

- 

7.32 

0 

0 

0 

0 

0.125 

0 

0.037 

n 

- 

- 

7.33 

0 

0 

0 

0 

0 

0 

0 

0 

V 

- 

7.35 

0 

0 

0 

0 

0.200 

0 

0.333 

0.034 
/II 

0 

n 
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ABLE  E.8:    JACCARD  COEFFICIENTS  OF    FAILURE  REGIONS  OF 

VERSION  vin 


8.1 

8.2 

8.3 

8.4 

8.5 

8.6 

8.7 

8.8 

8.1 

in 

- 

- 

- 

- 

- 

- 

- 

8.2 

0 

I 

- 

- 

- 

- 

- 

- 

8.3 

0 

0.500 

IV 

- 

- 

- 

- 

- 

8.4 

0 

0.143 

0.250 
/IV 

IV 

- 

- 

- 

- 

8.5 

0 

1.00 

0.500 

/IV 

0.429 

/IV 

IV 

- 

- 

- 

8.6 

0 

0.500 

0/IV 

0.125 
/IV 

0.500 
/IV 

IV 

- 

- 

8.7 

0 

0.333 

0.667  /IV 

0.111  /IV 

0.333  /IV 

0.667  /IV 

IV 

- 

8.8 

0.250 

0 

0 

0 

0 

0 

0 

V 

8.9 

0 

0 

0 

0 

0 

0 

0 

0 

8.10 

0 

0 

0 

0 

0 

0 

0 

0/V 

8.11 

0 

0.333 

0.200 

0.100 

0.250 

0.200 

0.167 

0.667 

8.12 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

8.13 

0.750  /III 

0 

0 

0 

0 

0 

0 

0.500 

8.14 

0.222 

0 

0 

0 

0 

0 

0 

0 

8.15 

0.750 

0 

0 

0 

0 

0 

0 

1.00 

8.18 

0.143 

on 

0 

0 

0 

0 

0 

0 

8.19 

0.286 

0 

0 

0 

0 

0 

0 

0.500 

8.23 

0 

0.500/1 

0.333 

0.111 

0.333 

0.333 

0.250 

1.00 

8.25 

0 

0 

0 

0 

0 

0 

0 

0 

8.26 

0 

0 

0 

0 

0 

0 

0 

0 
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TABLE  E.8:   JACCARD  COEFFICIENTS  OF   FAILURE  REGIONS  OF 

VERSION  vni 


8.9 

8.10 

8.11 

8.12 

8.13 

8.14 

8.1 

5 

8.1 
8 

8.1 
9 

8.2 

3 

8.2 

5 

8.2 
6 

8.9 

n 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.10 

0 

V 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.11 

0 

OA^ 

V 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.12 

0 

0 

0 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

8.13 

0.16 
7 

0 

0 

0 

m 

- 

- 

- 

- 

- 

- 

- 

8.14 

0.09 
l/II 

0 

0 

0 

0.12 
5 

n 

- 

- 

- 

- 

- 

- 

8.15 

0.20 
/II 

0 

0 

0 

1.00 

0.429 
/II 

n 

- 

- 

- 

- 

- 

8.18 

0 

0 

0 

0 

0 

0.100 

0.250 

I 

- 

- 

- 

- 

8.19 

0.125 

/n 

0.167 

0.167 

0 

0.400 

0.2 
/II 

0.75 

/n 

0 

n 

- 

- 

- 

8.23 

0 

0.50 

0.50 

0 

0 

0 

0 

O/I 

0 

I 

- 

- 

8.25 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

VI 

- 

8.26 

O/II 

0 

0 

0 

0 

OAI 

0  /n 

0 

0 

0 

0.667 

n 
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TABLE  E.8:  JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  Vffl 


8.1 

8.2 

8.3 

8.4 

8.5 

8.6 

8.7 

8.8 

8.9 

8.10 

8.27 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.28 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.29 

0 

0 

0/IV 

OIV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.30 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.31 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.32 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.33 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.34 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 

8.35 

0 

on 

0 

0 

0 

0 

0 

0 

0 

0 

8.36 

0.333 

0 

0 

0 

0 

0 

0 

0.143 

0.091 

/II 

0 

8.37 

0.600 

0 

0 

0 

0 

0 

0 

0.333 

0.143 

/n 

0 

8.38 

0.600 

0 

0 

0 

0 

0 

0 

0.333 

0.143 
/II 

0 

8.39 

0 

0 

0 

0 

0 

0 

0 

0 

O/II 

0 

8.40 

0.500 

0 

0 

0 

0 

0 

0 

0.250 

0.125 
/II 

0 

8.41 

0.500 

0 

0 

0 

0 

0 

0 

0.333 

0.143 
/II 

0 

8.42 

0 

0 

0.500 

0.143 

1.00 

0.500 

0.333 

0/V 

0 

0/V 

8.44 

0.333 

0 

0 

0 

0 

0 

0 

0.167 

/v 

0.100 

0/V 

8.45 

0.200 

1.00 

0.333 

0.125 

0.500 

0.333 

0.250 

0/V 

0 

0/V 

8.46 

0.333 

0 

0 

0 

0 

0 

0.333 

0.333 

/v 

0.200 

0/V 

8.48 

0.5/III 

0 

0.500 

0.143 

1.00 

0.500 

0 

0 

0 

0 

8.50 

0 

0 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0 

0 

0 
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TABLE  E.8:    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vm 


8.11 

8.12 

8.13 

8.14 

8.15 

8.18 

8.19 

8.23 

8.25 

8.26 

8.27 

0 

OAV 

0 

0 

0 

0 

0 

0 

0.500 

/iv 

0.500 

8.28 

0 

OAV 

0 

0 

0 

0 

0 

0 

0.667 

/iv 

0.667 

8.29 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0.400 
/IV 

0.400 

8.30 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0/IV 

0 

8.31 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0/IV 

0 

8.32 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0/IV 

0 

8.33 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0/IV 

0 

8.34 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0/IV 

0 

8.35 

0 

0 

0 

0 

0 

on 

0 

on 

0 

0 

8.36 

0 

0 

0.250 

0.333 

m 

0.571 

/n 

0.1 

0.3 

/II 

0 

0 

O/II 

8.37 

0 

0 

0.50 

0.222 

/II 

i.o/n 

0.167 

0.33- 
3/II 

0 

0 

O/II 

8.38 

0 

0 

0.500 

0.222 
/II 

1. 00/11 

0.167 

0.33 

/n 

0 

0 

O/II 

8.39 

0 

0 

0 

O/II 

O/II 

0 

o/n 

O/II 

0 

O/II 

8.40 

0 

0 

0.400 

0.30- 
O/II 

1. 00/11 

0.143 

0.42- 
9/II 

0 

0 

O/II 

8.41 

0 

0 

0.500 

0.333 
/II 

O/II 

0.167 

0.50 
/II 

0 

0 

O/II 

8.42 

0.333/V 

0 

0 

0 

0 

0 

0 

0.50 

0 

0 

8.44 

0/V 

0 

0.286 

0.250 

0.667 

0.111 

0.429 

0 

0 

0 

8.45 

0.250/V 

0 

0 

0.125 

0.500 

0.25 

0 

0.33 

0 

0 

8.46 

o/v 

0 

0.500 

0.333 

1.00 

0 

0.500 

0 

0 

0 

8.48 

0.333 

0 

O/III 

0 

0 

0 

0 

0.500 

0 

0 

8.50 

0 

0/IV 

0 

0 

0 

0 

0 

0 

0/IV 

0 
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TABLE  E.8:    JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vra 


8.27 

8.28 

8.29 

8.30 

8.31 

8.32 

8.33 

8.27 

IV 

- 

- 

- 

- 

- 

- 

8.28 

0.500  /IV 

IV 

- 

- 

- 

- 

- 

8.29 

0.333/IV 

0.400/IV 

IV 

- 

- 

- 

- 

8.30 

0/IV 

0/IV 

0/IV 

IV 

- 

- 

- 

8.31 

0/IV 

0/IV 

0/IV 

0.444/IV 

IV 

- 

- 

8.32 

0/IV 

0/IV 

0/IV 

0.667  /TV 

0.500  /IV 

IV 

8.33 

0/IV 

0/IV 

0/IV 

0.667  /TV 

0.500  /IV 

0. 800/1 V 

IV 

8.34 

0/IV 

0/IV 

0/IV 

0.364  /IV 

0.308/IV 

0.400 /TV 

0.400  /IV 

8.35 

0 

0 

0 

0 

0 

0 

0 

8.36 

0 

0 

0 

0 

0 

0 

0 

8.37 

0 

0 

0 

0 

0 

0 

0 

8.38 

0 

0 

0 

0 

0 

0 

0 

8.39 

0 

0 

0 

0 

0 

0 

0 

8.40 

0 

0 

0 

0 

0 

0 

0 

8.41 

0 

0 

0 

0 

0 

0 

0 

8.42 

0 

0 

0 

0 

0 

0 

0 

8.44 

0 

0 

0 

0 

0 

0 

0 

8.45 

0 

0 

0 

0 

0 

0 

0 

8.46 

0 

0 

0 

0 

0 

0 

0 

8.48 

0 

0 

0 

0 

0 

0 

0 

8.50 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 

0/IV 
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TABLE  E.8:   JACCARD  COEFFICIENTS  OF  FAILURE  REGIONS  OF 

VERSION  vin 


8.34 

8.35 

8.36 

8.37 

8.38 

8.39 

8.40 

8.41 

8.42 

8.44 

8.45 

8.46 

8.48 

8.5 
0 

8.3 
4 

IV 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.3 

5 

0 

I 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.3 
6 

0 

0 

n 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.3 
7 

0 

0 

0.42 

/II 

n 

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.3 
8 

0 

0 

0.4- 
3/II 

1.0 

/II 

n 

- 

- 

- 

- 

- 

- 

- 

- 

- 

8.3 
9 

0 

0 

O/II 

O/II 

O/II 

n 

- 

- 

- 

- 

- 

- 

- 

- 

8.4 
0 

0 

0 

0.5- 
O/II 

0.75 
/II 

0.75 

0/n 

0 

/II 

n 

- 

- 

- 

- 

- 

- 

- 

8.4 

1 

0 

0 

0.44 
4/II 

0.60 
/II 

0.60 

/n 

O/I 
I 

0.83 
3/II 

n 

- 

- 

- 

- 

- 

- 

8.4 

2 

0 

0 

0 

0 

0 

0 

0 

0/ 
V 

V 

- 

- 

- 

- 

- 

8.4 

4 

0 

0 

0.36 
4 

0.37 

5 

0.37 
5 

0 

0.44 
4 

0.5- 
0/V 

0/ 

V 

V 

- 

- 

- 

- 

8.4 
5 

0 

0 

0.12 
5 

0.25 
0 

0.25 
0 

0 

0.20 
0 

0.2- 
5/V 

1.0/ 
V 

0.1- 

4- 

3/V 

V 

- 

- 

- 

8.4 

0 

0 

0.33 
3 

0.40 
0 

0.40 
0 

0 

0.50 
0 

0.6- 

0/ 
V 

0.3- 

75- 

/v 

0.6- 
67- 

/v 

V 

- 

- 

8.4 
8 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1.0 

0 

m 

- 

8.5 
0 

0.08 
3/IV 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

IV 
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